[{"content":"This site used to be durkode.com. I have now switched it to ha.mi.sh, with the sole reason being I got the domain and I\u0026rsquo;ll be damned if I\u0026rsquo;m not going to use it.\nThat is all.\n","permalink":"https://ha.mi.sh/durkode-ha-mi-sh/","summary":"\u003cp\u003eThis site used to be durkode.com. I have now switched it to ha.mi.sh, with the sole reason being I got the domain and I\u0026rsquo;ll be damned if I\u0026rsquo;m not going to use it.\u003c/p\u003e","title":"Durkode -\u003e ha.mi.sh"},{"content":"I\u0026rsquo;m Hamish Durkin, a software infrastructure engineer from Sydney, Australia. I\u0026rsquo;ve spent the last several years at Canva working on infrastructure.\nThis blog is a place to publish my thoughts independent of any social platform. I make no promises regarding the frequency, quality, or thematic consistency of what I write here. Rather it is a collection of my thoughts when the inspiration strikes to record and share them. This site is as much for me as it is for you.\nTo get in touch, message me on LinkedIn or send me an email at hamish@durk.in. Given how sporadically I write here, I\u0026rsquo;m equally sporadic at replying — but I do read everything.\n","permalink":"https://ha.mi.sh/about/","summary":"\u003cp\u003eI\u0026rsquo;m Hamish Durkin, a software infrastructure engineer from Sydney, Australia.\nI\u0026rsquo;ve spent the last several years at Canva working on infrastructure.\u003c/p\u003e\n\u003cp\u003eThis blog is a place to publish my thoughts independent of any social platform.\nI make no promises regarding the frequency, quality, or thematic consistency of\nwhat I write here. Rather it is a collection of my thoughts when the inspiration\nstrikes to record and share them. This site is as much for me as it is for you.\u003c/p\u003e","title":"About"},{"content":"Today I got my Dell XPS 9350 working with 2 external monitors via an Aorus 1080 gaming box EGPU. There are some caveats, but for the guide read below (and skip over the reasoning section if you are only interested in the technical components).\nReasoning My desktop is old and beat up. i5-750 CPU first put together in high school nearly 10 years ago (mid 2009 if memory serves correctly). Back then things were a bit different. I had my desktop for nearly everything, my cheap as possible netbook for class (although lets be honest, I never really used it for that) and Dropbox to sync my files.\nIn 2019 the landscape is different. I have a personal laptop (13\u0026quot; Dell XPS 9350), work laptop (what was a top specced Macbook Pro 6 months ago), that same desktop with a few alterations but same core motherboard + CPU, and finally an intel NUC that is surplus to needs and still in storage.\nI also have 2 monitors (34\u0026quot; landscape and 27\u0026quot; portrait) on stands that are fantastic, although hooked up to the old desktop I never work on. The fantastic workspace is going to waste as I do all my work on my laptops. The compromise is that I would hook up a single laptop to a monitor, however that leaves one monitor unused and I have to then use my laptops keyboard + trackpad (or set up further more accessories). What would be ideal is the one stop dock.\nI looked in to getting a thunderbolt 3 dock such as https://www.elgato.com/en/dock/thunderbolt-3, however it creeps up to $400 landed, and still has limitations such having to drive the external monitors off my laptops integrated graphics. In addition, any desktop I build will have to have a titan ridge card such as https://www.gigabyte.com/Motherboard/GC-TITAN-RIDGE-rev-10#kf to use the dock. It is 2019 and the best solution is feeding the video output back into the thunderbolt card before output. No thanks if I can avoid it. Additionally, requiring that card in addition to a graphics card would require \u0026gt; 2 PCI slots (as decent GPUs are at least 2 slots), meaning no small factor mITX build.\nThe solution I settled on was to get an Aorus gaming box 1080. A bit pricey at around $1k AUD, however its a decent price when you consider the alternatives (a separate dock + GPU). It comes with 4 USB ports (1 power only) so you can plug in your keyboard and mouse and have a 1 cable setup in theory. Added bonus of this is I can install Windows on my currently unused Intel NUC and hopefully have a somewhat capable gaming setup.\nHow to set up This section is how I got it working. Later on I will tell you everything that didn\u0026rsquo;t work.\nInstall Ubuntu 18.04 LTS. 18.10 may also work, but I did not test. Ensure you are connected to the internet and install third party packages as it will save a lot of time later (probably optional, but I always do this). (Optional but recommended) Install standard system updates. There should be a pop up for system updater that prompts you to do this, and will ask for a reboot afterwards. Blacklist Nouveau drivers. I am unsure if this step is required, but I did it for safe measure (https://linuxconfig.org/how-to-disable-nouveau-nvidia-driver-on-ubuntu-18-04-bionic-beaver-linux) Plug in and authorise the EGPU. After plugging in, go to settings -\u0026gt; devices -\u0026gt; thunderbolt and you should see the graphics card listed. Ensure that Direct Access is turned to On and the device is Authorized. If you do not see your graphics card after it is powered on and plugged in to your laptop, it is likely a Bios issue. I didn\u0026rsquo;t encounter this as I had already configured my bios when I previously ran from thunderbolt port directly to my monitor. Install ubuntu drivers with sudo ubuntu-drivers autoinstall. Right now this should install nvidia-driver-390 and related libraries. Reboot This was actually surprisingly simple if you follow this golden path. However there are some catches.\nCaveats Graphics card is not used before Display Manager launches. That means you have to log in using your laptop screen, and then starts using the graphics card and your monitors No hotplugging. I could not get hotplugging to work. That means that pulling out the cable causes your system to freeze, and plugging in the cable causes nothing to happen until a reboot Lag on peripherals (supposedly). Only after I bought the gaming box did I read about USB lag when using the graphics card. I haven\u0026rsquo;t run workloads on the graphics card yet so haven\u0026rsquo;t experienced it yet. My keyboard feels a bit sluggish but my mouse is fine, so I guess that is just my mind playing tricks. I\u0026rsquo;m not gaming on Ubuntu so no big deal Can not seem to get gnome shell operational. My laptop was initially running gnome shell, and after installing the graphics drivers my system was detecting the plugged in monitors and X11 was allocating them desktop space, but the monitors were saying \u0026ldquo;no signal\u0026rdquo;. Only after reinstalling stock Ubuntu, then trying gnome shell was I able to recreate the issue, leading me to believe it is a gnome shell issue Failures I spent too long trying to get things (hotplugging, different drivers, early boot) going before giving up. Here are a few things I ran into:\nAs said above, gnome shell appears a no go. This isn\u0026rsquo;t a deal breaker for me given Ubuntu now essentially runs gnome since 18.04 I couldn\u0026rsquo;t pick and choose or use newer graphics. If I added the nvidia apt repository for newer drivers and installed them, my external monitors weren\u0026rsquo;t detected. Only nvidia-driver-390 seemed to do the trick I couldn\u0026rsquo;t manually install drivers with sudo apt-get install nvidia-driver-xxx. Same issue as above, but even trying driver 390 with this yielded problems. Rather than spending yet more time fighting this, I will let it be. In 3 years time when I need to reinstall my laptop OS, hopefully the state of things will be better I couldn\u0026rsquo;t get hotplugging working, despite others saying they were able to. In particular https://github.com/julianpoy/eGPUScripts seemed promising, although it just sent my laptop to a login loop. That\u0026rsquo;s not a dig at the author, I appreciate the effort of writing and publishing these scripts, but EGPU on Ubuntu is still new tech with rough edges. Summary I probably could spend more time ironing out the issues, but it is good enough as is for me. Even 1 year ago this would have been a monumental task as support only came with 18.04 and kernel 4.13, so I appreciate the work that all the devs have put into making it so relatively simple.\nNext up, I will try to get my work laptop working with this setup on MacOS. Stay tuned.\n","permalink":"https://ha.mi.sh/laptop-egpu-ubuntu-my-experience/","summary":"\u003cp\u003eToday I got my Dell XPS 9350 working with 2 external monitors via an Aorus 1080 gaming box EGPU. There are some caveats, but for the guide read below (and skip over the reasoning section if you are only interested in the technical components).\u003c/p\u003e","title":"Laptop + EGPU + Ubuntu. My Experience"},{"content":"The 2016 US Presidential election so far has been a splendid spectator affair. What was quite interesting to observe was Hillary Clinton having a recent health scare during a 9/11 memorial event. Now I am not one to be drawn into political debate, but I am one to analyse the odds around incidents such as these. Was there a dollar to be made?\nLooking at betfair, things get interesting. Hillary clinton is currently at $1.57 to take the presidency, but the democrats are currently at only $1.44. That seems like quite the discrepancy. It seems like the odds are suggesting that there is beyond a miniscule chance that the democrats will win without Hillary. What I want to do is bet against that notion, by backing Hillary to win but laying the Democrats in the event that Trump takes the victory.\nLets say I have $10,000 to risk on this bet. I can back Hillary at $1.57 ($14k on the table), and lay the Democrats at $1.46 (can lay currently at $1.45 but only $xxx, $1.46 has $11k on the table). Placing $4818 on Hillary and $5182 against yields $364 return (96.5%) unless a democrat other than Hillary is elected.\nBut then you need to take into fees. If you are slugged 5% on betfairs lower rung, that return drops to $185 (98.2%), or $292 (97.2%). Not as enticing.\nStill, that implies a ~3% chance that the US will elect a democrat other than Hillary to the top job in a vote held in less than 2 months. For that to happen:\nHillary needs to drop out. The only way I can see that happening is an assassination or health issue causing her to drop dead. The email server troubles seem to have passed, and nothing short of dropping dead before the election day will change this. Even if she has a serious health issue, she would soldier through to win for the democrats before handing over the reigns (this is just my personal opinion) Her replacement would need to win. This is a tough ask given the need to establish themselves as a candidate amid such controversy, and would greatly favour the republican. Therefore I would place the odds of that happening at less than 3%, and would entertain the bet. Alternatively if you hold that view, you can lay Joe Biden at $34 and Bernie Sanders at $65, although the volume isn\u0026rsquo;t there to put $10k on the line. Laying a mix of those two would probably be a safer option.\nIf I had a riskable $10k on tap I would consider it at the fee free rate, but once betfair fees on arbitrage between different markets comes in, it halves the profit and kills the incentive. Given my financial position I would consider max of $1k on this bet, however I feel with a less than 1% edge it is not worth the hassle. Also my lack of intimate knowledge of the US electoral system leaves possible unknown factors that makes me less comfortable.\nWhat started out looking as quite a large margin ($1.57 vs $1.46), once you take in fees and hedging both sides, doesn\u0026rsquo;t give the reward that I initially thought.\nJust had a look around. Ladbrokes is offering Hillary at 1.57. No commissions. No notion on liquidity, however based on past experience they try to limit exposure to $1,000. Could try placing the first few bets at that price\n","permalink":"https://ha.mi.sh/the-clinton-arbitrage/","summary":"\u003cp\u003eThe 2016 US Presidential election so far has been a splendid spectator affair. What was quite interesting to observe was Hillary Clinton having a \u003ca href=\"http://www.nbcnews.com/politics/2016-election/hillary-clinton-s-health-scare-9-unanswered-questions-n646551\"\u003erecent health scare\u003c/a\u003e during a 9/11 memorial event. Now I am not one to be drawn into political debate, but I am one to analyse the odds around incidents such as these. Was there a dollar to be made?\u003c/p\u003e","title":"The Clinton Arbitrage"},{"content":"Recently I read this article on how hackers obtained private press release and then sold this information to traders who then acted on them to make profitable trades.\nSeemingly the trading methodology was to buy out of money options that are assumed to be worthless just before the press release was made public, watch the value of these options skyrocket and then exercise or sell them.\nThe theoretical financial world is based on the concept of a purely rational market, in which information is perfectly distributed and actors act completely rationally. I have no experience in the financial world, however I would believe the opposite to be true in practice. It is probably filled with algorithms that follow the herd, and investors that trade in mischievous ways using insider knowledge.\nSo why don\u0026rsquo;t you create algorithms to exploit that? Program in press release times, look for unexpected or unexplainable trading activity across the different markets, and piggy back off the success of insider trading? Sure you might lose nearly as many as you gain, but if the presumption that there are people trying to earn their small fortune through insider trading holds true, I imagine this would be a profitable strategy. You might have to ask a lawyer about the legality of making a profit off trades hypothesised to be illegal.\nNow I know there are several problems with this idea. The first being that the insider trader has captured all of the gains. I wouldn\u0026rsquo;t imagine this would be the case, due to lack of capital, desire to not be caught, or inability to arbitrage all markets at once. In addition I couldn\u0026rsquo;t imagine an insider trader shifting the market so much as to overshoot the markets reaction to the information.\nThe second objection is that algorithms are already noticing the sudden moves and taking some of the profits as market maker. Your hidden advantage is pricing in the expected timing of future announcements into your algorithm to exploit this. Someone buying out of money options about to expire is normally crazy. Doing so when the expiry is soon after a major company announcement is a potential signal. Picking up on that signal is what will make this strategy work.\nThe third objection is that there is probably a hedge fund doing this already. And you\u0026rsquo;re probably right. I don\u0026rsquo;t have the time and/or resources to investigate this any further right now. It is merely a thought experiment on how the assumption of insider knowledge influencing the market could lead to strategies to exploit this fact. I\u0026rsquo;d be surprised if it worked, but most strategies are worth a shot.\n","permalink":"https://ha.mi.sh/how-to-make-money-from-insider-trading-without-insider-trading/","summary":"\u003cp\u003eRecently I read \u003ca href=\"http://www.bloombergview.com/articles/2015-08-11/why-not-insider-trade-on-every-company-\"\u003ethis article\u003c/a\u003e on how hackers obtained private press release and then sold this information to traders who then acted on them to make profitable trades.\u003c/p\u003e\n\u003cp\u003eSeemingly the trading methodology was to buy out of money options that are assumed to be worthless just before the press release was made public, watch the value of these options skyrocket and then exercise or sell them.\u003c/p\u003e","title":"How to make money from insider trading ... without insider trading"},{"content":"Many people have told me that you can gain everything you want in life, you just can\u0026rsquo;t have it all at the same time. It is said that when you are young you have time and energy but no money, in middle age you have money and energy but no time, and in old age you have money and time but no energy.\nThis makes me think - how long do I have left on this earth? When are these stages of life?\nA simple way of looking at this is by life expectancy. I am 21, and the average Australian life expectancy is 82. Going off this we can conclude that I have lived 1/4 of my life so far. No worries, I have heaps of time left.\nBut it\u0026rsquo;s not that straight forward. I remember my primary school days, when the days stretched out for eternity and the seasons felt as if they were locked in permanence. I remember when a 2 hour car trip was an unbearable ordeal, but now it goes in the blink of an eye. Perhaps how we perceive time is relative to our age? After all, every year seems to go quicker than the last.\nLet\u0026rsquo;s say year 1 is the base unit of time. You have no conception of how long is ahead of you or when a year will end. Year 2 is another measure of year 1, as it is a repeat of that base unit of time. Year 3 however, only feels like half as long as that first year, as it is half the length of time experienced thus far. Year 4 feels like a quarter.\nThus S(N) = 1 + 1 + 1/2 + 1/3 + \u0026hellip; + 1/(n-1) years, where S(N) is how long your life of N years is perceived in terms of your first year of life.\nS(82) ~= 4.977. That sounds stupidly small. But it isn\u0026rsquo;t. Do you actually remember your first year of life? If you did, maybe you would remember time moving that creepingly slow for you. However since no-one can remember their first year, it is an impractical measure.\nHow about measuring against something you can relate to? What about your first 10 years of life?\nIn that case, S(N) = 1 + 1/10 + 1/11 + \u0026hellip; + 1/(n-1).\nSo how quickly did the next 10 years of your life, S(20) - S(10), go by? As S(20) = (1 + 1/10 + 1/11 + \u0026hellip; + 1/19) ~= 1.718, therefore (S(20) - S(10))/S(10) ~= 0.718 . Therefore the second decade of your life went by in only 70% of the time of your first decade.\nUsing the decade (arbitrary decision) as the base perspective of time, how much of my time on this earth have I consumed?\nThat would be S(21) / S(82) ~= 1.769 / 3.149 = 56%. That is remarkably scary. If time were perceived linearly, that would mean I really only have until I am 38. That is remarkably soon.\nBut using the first decade is an arbitrary decision. The smaller the period of reference, the less perceived time we have left. As the period of reference approaches 0, so does the perception of any later period relative to it.\nThe best answer to this issue is to use your current age as the base reference. After all, it has encompassed everything you have perceived thus far. In doing so, S(21) / S(82) = 1 / (1 + 1/21 + 1/22 + \u0026hellip; + 1/81) ~= 42%. That means I have lived 42% of my perceived life.\nOn an interesting note, if you are expected to live until 82, you have lived half your life at age 30. Must be why no-one wants to turn 30.\nRunning these sums, the result backs up my own intution on how I perceive time. On the downside, I have less time to do what I want to do. On the plus side, these numbers remind me not to waste what dwindling time I have.\n","permalink":"https://ha.mi.sh/time-keeps-accelerating/","summary":"\u003cp\u003eMany people have told me that you can gain everything you want in life, you just can\u0026rsquo;t have it all at the same time. It is said that when you are young you have time and energy but no money, in middle age you have money and energy but no time, and in old age you have money and time but no energy.\u003c/p\u003e","title":"Time Keeps Accelerating"},{"content":"I don\u0026rsquo;t want this post to turn out with that soppy, transcendental tone that I read so often and slightly cringe at. I feel I just had an important experience that reinforces a recent change in attitude towards maths and its role in my work.\nI have always been driven by the pursuit of pragmatism. Tell me an idea, and I will ask where I can use it. This has led to an interesting relationship between maths and myself.\nDuring early high school I had a natural aptitude for maths. I remember in year 7 I placed in the top 0.1% of the country for a national maths contest. I got a nice $50 check and the feeling I pulled off a great fluke. In school I always felt maths was the easiest subject and relatively straightforward, but over time my advantage in the subject relative to my peers fell off for two reasons. Firstly, I was a lazy individual who subscribed to the theory of \u0026lsquo;maximum marks, minimum effort\u0026rsquo;. Secondly, the carrot the school hung in front of us was changing from a natural interest in maths towards a pursuit to increase the trailing digits in the final mark that the government says sums up all your achievements in education. Thankfully I didn\u0026rsquo;t want to be a doctor or a lawyer, so I couldn\u0026rsquo;t care less about squeezing the rightmost digits of that score.\nI hoped university would instill a newfound love of maths, but this was not the case. All that had changed was that now the lecturers were interested in demonstrating their academic superiority by explaining relatively simple concepts in rigorously defined and tested terms that lost all meaning to my practical mind. There were exceptions to this (thank you to Peter Brown, my Finite lecturer) however my mind was settled on the matter.\nIn my first year of university, 5 out of the 9 courses I studied were maths courses. That might sound like a love of maths, but it was a desire to get it done with as quickly as possible.\nMaths 1A (Linear Algebra / Calculus) Maths 1B ' Discrete Maths Finite Maths (fields, number theory, error-correcting codes) Probability, Stats and Information I did the bare minimum to skate through, but managed to get distinctions in most of them and a HD in Finite. On paper, I was a great student who could demonstrate an understanding on all the topics discussed. In reality, I had put my many years of high school examination preparation into practice to hone in exactly on the style of questions that I knew would surface. It really is an extroadinary skill I had developed, to know exactly what I was required to know and not waste my cranium on the remainder.\nWhile I did apply some maths knowledge in other domains, the problems were usually of such simplicity that I attributed any success to common sense, or so contrived that I felt I was being tricked into seeing the usefulness of it. If I wanted to discover the applications of maths for my field of work, it wasn\u0026rsquo;t possible for someone to show me, I had to find it out for myself.\nIn S1 2014 I studied AI. No disrespect to Alan Blair (lecturer, now my thesis supervisor), but the first half of that course I found utterly disinteresting. All credit to Alan, the last half of the course contained some of the most interesting things I had learnt at university. Decision trees, Neural Networks, Deep Learning. I was inspired. There was little discussion of the implementation, I knew I had to use a sigmoid function and calculus to facilitate backpropogation, but surely someone else had wrapped that in a nice little box for me and that would be that.\nFast forward to the last 6 months. I have elected to do a thesis on Robust Recommeder systems (collaborative filtering algorithms resistant to collusion). I knew I liked the idea of machine learning, but I didn\u0026rsquo;t know what I was getting myself into.\n\u0026ldquo;What is that?\u0026rdquo; \u0026ldquo;A vector, why is that a vector\u0026rdquo; \u0026ldquo;They\u0026rsquo;re all vectors\u0026rdquo; \u0026ldquo;How do I multiply a matrix and a vector again\u0026rdquo; \u0026ldquo;L-2 Norm, that sounds complex, I\u0026rsquo;ll come back to that\u0026rdquo; \u0026ldquo;I see the algorithm, but what am I trying to minimise?\u0026rdquo; \u0026ldquo;Wait, that\u0026rsquo;s inverse?? I thought that was transposed??\u0026rdquo; \u0026ldquo;WHY ARE THEY ALL VECTORS, please just give me scalars, I like single numbers, they are easy and logical\u0026rdquo;\nYeah, I freaked out a little bit. But slowly I have been picking it all up again. In an academic pursuit. Slowly but surely I am learning my first year maths again and gaining back my linear algebra intuition. But still, I don\u0026rsquo;t plan on going into research, someone else will make a library for me to do this when I go into the wild.\nThe Practical Application At work last week, I had to create a program that makes predictions. I won\u0026rsquo;t go into details about for what purpose, but it involved using linear regression on a small sample of datapoints, but this method was applied to thousands of different instances (with the hope to scale to millions later).\nEasy I say. I grab Apache Spark\u0026rsquo;s MLLIb, solve the problem with Linear Regression powered by SGD. Done.\nOh no, MLLib is overkill for such a small amount of datapoints per instance (\u0026lt;100), let me try another library. Great, Apache Commons Maths has an OLS regression implementation. No mucking around with gradient descent or setting parameters for that. Just a mathematically pure minimisation.\nOh no, the model grossly overfits. The pursuit for a mathematically pure minimisation fit the data well but was a horrible model.\nNo worries. I\u0026rsquo;ll just use a regularised version of this. Let me just open up the libarary \u0026hellip; wait, it\u0026rsquo;s not here. Cue me spending several hours scouring the internet to find an appropriate library to use.\nFinally I thought I would take the plunge. I looked up the appropriate wikipedia article and got to work convincing myself the equation was correct, what all the variables meant, how it was derived, etc. I implemented it in 10 minutes in Scala, and it worked exactly as I expected. There was an inherent beauty in the solution, and the fact that a few seemingly unrelated and abstract matrix operations could combine with the help of calculus to provide a model for the data.\nThis may not be earth shattering, but I feel it is the first time I have used non-trivial university level maths to solve a real problem I have had. It felt good. I was able to solve a problem with no library to do it for me, and that I understood mathematically exactly what the algorithm was doing and how it was derived.\nIt is a thing of beauty when such abstract concepts combine to deliver a result that adds real value to our world. Hopefully I\u0026rsquo;ll be more aware of when these situations arise in the future, and be more eager to solve them myself (when appropriate) rather than spend time searching for a library that will be \u0026lsquo;good enough\u0026rsquo;\n","permalink":"https://ha.mi.sh/a-new-appreciation-for-maths/","summary":"\u003cp\u003eI don\u0026rsquo;t want this post to turn out with that soppy, transcendental tone that I read so often and slightly cringe at. I feel I just had an important experience that reinforces a recent change in attitude towards maths and its role in my work.\u003c/p\u003e","title":"A New Appreciation For Maths"},{"content":"My 6 month journey through to Sweden provided much food for thought. Honestly, while I was over there I wasn\u0026rsquo;t thinking so much, but when I returned to Australia the holiday was over and I knew some of those dreaded \u0026rsquo;life decisions\u0026rsquo; would need to be made.\nWhat struck me as bizare about Europe was the social norm that young people are expected to be students until they are in their mid to late twenties. Over there, you have not completed university until you have a postgraduate degree and finding relevant work without one was difficult at best. I believe this is a function of two factors: the poor economic situation (high youth unemployment) causing students to remain in institutions for longer, as well as a stronger cultural demand for a liberal education.\nThis prompted two distinct reactions from me.\nFirstly, I am glad to live in Australia where there is no pressure to receive postgraduate degrees. I can choose to enter the workforce in valued professions and succeed with only an undergraduate degree. This is a freedom that is not as prevalent in Europe. I do not believe that in Australia the barrier to success is lower and further education is not required, just that there are other paths than earning a second piece of paper. I\u0026rsquo;m young, I shouldn\u0026rsquo;t be thinking about my second degree before the first one is finished.\nMy second reaction was to contemplate a second degree. Up until that point, my thoughts on the matter were simple: I wanted to work. I, along with nearly all students in the computer science school, want to take advantage of the high demand for graduates in our field and have great lives doing important things for companies in highly demanding, transformative positions. The thought of going back to university never even crossed my mind, except maybe doing it once I am 30 and looking to brush up on business qualifications for more senior managerial positions (how depressing does that sound to 21 year old me). My view was that technical (i.e. non arts / MBA) postgraduate courses straight out of university are the breeding ground for career academics, and that is a path I firmly do not want to head down. I have great respect for the many academics I know, however with the head-strong surety that my youth provides I am certain that it is not the path for me.\nBut life in Europe began to change my view on secondary degrees. People my age were finishing their undergraduate degrees in preparation for their masters. Combined with this, many things I felt knew to be certain previously were becoming less certain. I feel that I now know less about my industry than I did two years ago. This is not due to knowledge being forgotten or becoming obsolete, but rather due to losing my ignorance about what I don\u0026rsquo;t know.\nThis diminished sense of knowledge was also accompanied by a blurry sense of direction. In first year this sense of direction was crystal clear:\nThem: \u0026ldquo;So what do you study?\u0026rdquo;\nMe: \u0026ldquo;I\u0026rsquo;m in my first year of Software Engineering at UNSW\u0026rdquo;\nThem: \u0026ldquo;How are you liking it?\u0026rdquo;\nMe: \u0026ldquo;Pretty good. Courses are easy, job prospects are good. I\u0026rsquo;ll pick up a job as a rockstar ninja developer, and if my projections are correct, with the increased demand for developers and the deteriorating European economy, I should be able to retire in 10 years and buy 1/3 of Greece\u0026rdquo;\nBut alas young naivety has been knocked out of me and the conversation has changed now that I am no longer a freshling.\nThem: \u0026ldquo;So what do you study?\u0026rdquo;\nMe: \u0026ldquo;I\u0026rsquo;m in my fourth year of Software Engineering at UNSW\nThem: \u0026ldquo;So what are you specialising in and what are your plans for next year?\u0026rdquo;\nMe: \u0026ldquo;Well I haven\u0026rsquo;t really thought about specialising, I\u0026rsquo;m a generalist who has put my hand in all the jars but nothing really stuck? As for next year, I don\u0026rsquo;t know, maybe something to do with the web?? I hear that\u0026rsquo;s pretty big in computing?\u0026rdquo;\nThem: \u0026ldquo;Web dev? Front end or backend? Ops? Architect? Startup or enterprise? Sydney or overseas? Or perhaps more university?\u0026rdquo;\nPostgraduate degrees made their return into the decision tree from which they had previously been banished and now my future now has more options than ever. This is equally liberating and daunting, as I am at the end of the common lines of the opening book. While there are some mainstream variations to know, this is now about midgame tactics and strategy to achieve goals that I have yet to set.\nSo the question is \u0026ldquo;what do I do now?\u0026rdquo; And there is no right or wrong answer, just different shades of grey. My answer to that question is not a constant, but rather a function of time and circumstance that moves in trends. Some paths have been made clear to me, and I am sure others will emerge over time. They range from the academic to the hands on, and here are some of them:\nJoin a large non-tech enterprise and deliver solutions for their business needs. Even after removing the jargon this sounds dirty, with all the networking and meetings that come with the territory. This would involve gaining further technical and business qualifications Join a large tech focused company and solve complex problems with a deep understanding in a limited domain whilst working with brilliant people. This would involve a postgraduate level of education in a specific field. Join a young tech company and relish the breadth of knowledge required and fast iteration that comes with it. No formal training required, but I will be directing my own learning. Start my own business, be my own master, conquer the world, be filled with unbridled optimisim and yada yada yada. Experience is the best education. None of these answers are absolutes, they are merely destinations on a map I am trying to make a little less blurry. If I choose one of these destinations, probability says that I will adjust course halfway through my journey and find myself at other locations. Some of these destinations are in Australia, but others are better reached overseas. I have no intention of making a map that I follow religiously for the next 20 years (believe me, I have seen several people try that and regret it after only 2-3 years), but I would love to at least pick a destination and get on the road rather than sitting at home doing nothing.\nWith that in mind, I still have been unable to pick a specific destination. However I have picked a direction to head in that leads to several destinations in the hope that I will know which fork to take when I reach it. I have decided to focus in on AI, machine learning, big data, cloud technology, and various other buzzwords. It is a group of general fields that are interesting to me, complementary, yet at the same time definitely a step in a direction. They bring back the proposition of postgraduate education, but it is not a necessity. Now the conversation goes like this:\nThem: \u0026ldquo;So what do you study?\u0026rdquo;\nMe: \u0026ldquo;I\u0026rsquo;m in my fourth year of Software Engineering at UNSW\nThem: \u0026ldquo;So what are you specialising in and what are your plans for next year?\u0026rdquo;\nMe: \u0026ldquo;I am studying machine learning with a focus on cloud and distributed computing, relevant for big data analysis\u0026rdquo;\nThem: \u0026ldquo;And what sort of role would that lead to\u0026rdquo;\nMe: \u0026ldquo;It varies. It could be enterprisy, it could be consulting, it could be growth hacking for a startup, it could mean back to university for a few more years. I don\u0026rsquo;t know yet but I\u0026rsquo;ll navigate that fork when I get there\u0026rdquo;\nIt doesn\u0026rsquo;t matter that I don\u0026rsquo;t know the destination. What matters is that I made a step in a promising direction, which is something I have wanted to do for several years now.\nI used to believe that the destination was what mattered, that when you reach the destination it was game over. Stop doing, stop learning, stop adapting, game over. Now I see the journey as the most important aspect, and the only use of the destination is to know what steps to make on the journey.\n","permalink":"https://ha.mi.sh/working-learning-and-the-path-ahead/","summary":"\u003cp\u003eMy 6 month journey through to Sweden provided much food for thought. Honestly, while I was over there I wasn\u0026rsquo;t thinking so much, but when I returned to Australia the holiday was over and I knew some of those dreaded \u0026rsquo;life decisions\u0026rsquo; would need to be made.\u003c/p\u003e","title":"Working, Learning, and the Path Ahead"},{"content":"I have caught a piece of World Cup Fever, and although Australia may not have the easiest route forward (understatement of the year) the tournament has been lively and enjoyable so far.\nI have been following the betting markets, mainly as I view it as the most accurate predictor of how the world cup will unfold. Having said that, there have been many upsets already, such as the Dutch thumping of Spain and Costa Rica\u0026rsquo;s amazing defeat of Uruguay. I have been monitoring the markets closely as I am interested in developing betting systems in the near future.\nI know many people believe that gambling is a sinkhole of money, but I believe that is only true in the traditional sense of taking a punt. With arbitrage based methods, profit is guaranteed if the execution is successful. With predictive models, expected profit can be calculated through extensive backtesting.\nNow my arbitrage example: Who will be the winner of Group H in the world cup - Belgium, Russia, South Korea or Algeria?\nSportsbet will payout $26 on Algeria. However on Betfair, you can Lay Algeria at $21. To Lay a bet means to act as a bookmaker, i.e. betting against an event to happen. Let\u0026rsquo;s do the calculations to work out how much to put down:\nLet x = amount backing Algeria at sportsbet Let y = amount against Algeria at Betfair Let R = Return (we want it to be the same, win or lose)\nAlgeria wins: R = 26x - 21y\nAlgeria Loses: R = 0.95y - x (as Betfair takes a max of 5% comission on wins)\nTherefore: x = 0.95y - R R = 26(0.95y - R) - 21y 27R = 3.7y R = 3.7y / 27\nTherefore, if you lay $200 on Algeria (near market depth at time of writing, used for simplicity sake), you should bet $162.59 on Algeria at Sportsbet, resulting in a profit of $27.40 no matter the result. That seems like a decent return.\nWell that would be jumping to conclusions too quickly. The above is all true, that if you place the bets as described above you are guaranteed to make $27.40. What it doesn\u0026rsquo;t reveal is the liability you need to be prepared to cover if Algeria wins.\nTotal capital to make the above bets is $162.59 in the Sportsbet account, and $200 x 21 = $4362.59. Therefore return is now (27.40 / 4362.59) * 100 = 0.62%. Annualised, given that the last game is on the 27th of June, 12 days away, the annualised return is 0.62 * (365/12) = 30.4%. Better, but still not worth it given such a low absolute return for the effort and risk (i.e. successful exeuction of orders) required.\nWhile this particular arbitrage opportunity is not what it seems, I feel that there are good opportunities in the market in other arenas. While bots tend to dominate in arbitrage, I think there are still some novel approaches that could yield fruit.\nI may try building some arbitrage tools later on, however the high bankroll required and risk in making the executions make it less appealling to me. I feel that predictive modelling has the potential to yield better results.\n","permalink":"https://ha.mi.sh/world-cup-arbitrage/","summary":"\u003cp\u003eI have caught a piece of World Cup Fever, and although Australia may not have the easiest route forward (understatement of the year) the tournament has been lively and enjoyable so far.\u003c/p\u003e","title":"World Cup Arbitrage"},{"content":"I was recently looking for a way to download the latest version of spotify on my Ubuntu machine (hint: it\u0026rsquo;s not release yet and probably won\u0026rsquo;t be for a while), when I ran across the following thread on the spotify forum:\nSpotify P2P uploading and cache\nThe gist of the thread is that Spotify employs a hybrid download model, of both p2p connections when available and otherwise direct streaming from its servers. However this fact is never advertised to the user. I was quite shocked to discover this, despite having been a user of Spotify for several years. If this was something that was limited to the free version I could understand, however p2p seems to be a part of the paid service too (I haven\u0026rsquo;t yet inspected the traffic myself, however other reports online agree).\nWhile this doesn\u0026rsquo;t affect me personally (not a gamer and uploads don\u0026rsquo;t count to my quota), I can certainly sympathise with those that it does affect. Firstly those who are connected via a service with low data limits with uploads counted, such as on a 3g modem or tethered to their phone. These users often have poor upstream bandwidth, compounding the problem. Secondly, there are gamers, skypers and other latency sensitive users that will have a poor experience and often not know why. It is obvious knowledge to disable your torrent client whilst using these application, however I have never once thought to turn off spotify whilst it is running in the background.\nShould applications be required to disclose that they use p2p networking to deliver content? Gaming company ESEA was fined $1 million for hiding a bitcoin miner in their software, in which the company profited from stealing unutilised cpu (or gpu) cycles from end users\u0026rsquo; computers without disclosure. Should Spotify face similar sanctions for profiting from using excess upstream bandwidth on clients\u0026rsquo; network connections?\nI won\u0026rsquo;t stop using Spotify over this as love it too much and haven\u0026rsquo;t found a comparable service here in Australia. However by including this \u0026lsquo;feature\u0026rsquo; for paying customers without any way to disable or control it, Spotify has given me one large reason to switch should the opportunity arise.\n","permalink":"https://ha.mi.sh/spotify-p2p-the-unadvertised-feature/","summary":"\u003cp\u003eI was recently looking for a way to download the latest version of spotify on my Ubuntu machine (hint: it\u0026rsquo;s not release yet and probably won\u0026rsquo;t be for a while), when I ran across the following thread on the spotify forum:\u003c/p\u003e","title":"Spotify P2P - The Unadvertised Feature"},{"content":"Earlier this week we had a guest lecturer for our Software Engineering Workshop course (SENG3011 for those at UNSW). The guest was Andrew Maxwell from Optiver, a Dutch trading firm that has a strong Australian presence and invests heavily in IT solutions to facilitate their business (both for algorithmic traders and their human counterparts).\nUsually guest lecturers are far more interesting than their internal counterparts as they bring an applied perspective and experience, and Andrew was no exception. While there was not enough time to go into depth on their operations, he gave a broad outline of Optiver\u0026rsquo;s approach to both trading and the software development behind it. While this was interesting, what stuck with me was the recurring theme that reality and theory often disagree with each other.\nYou might be thinking \u0026ldquo;Well that sounds extremely obvious\u0026rdquo; and I would agree with you, but when combined with the examples from Andrew\u0026rsquo;s previous work it provided a lot of food for thought. Initially he applied this principle when discussing varying algorithms, and how different worst, average and best case complexities were sometimes useless in making decisions, as the size and distribution of data rarely lined up with any of these scenarios.\nThe next example was of a team of recent Ivy league mathematics/CS graduates that had created a sophisticated machine learning algorithm for algorithmic trading. The project was billed as a money making machine. The only problem was that when they implemented it, the algorithm was so complicated that they couldn\u0026rsquo;t understand why the program was making the decisions that it did, and when the market turned so did the profits.\nMost sticking of all, was the following statement:\nTheoretically, Optiver shouldn\u0026rsquo;t exist\nOptiver trades equities not based on their projected future value, but on what their current value is. However according to free market theory, the \u0026lsquo;invisible hand\u0026rsquo; of the market will price everything in the market at its actual value. Therefore, how can Optiver make a profit if everything is already priced at its true market value?\nThe answer is that the in order for the \u0026lsquo;invisible hand\u0026rsquo; to operate, market makers are required to continuously buy and sell to keep the price of goods and services in line with their value. Optiver is one of those market makers, and I\u0026rsquo;m sure they earn a hefty chunk of change in this capacity.\nThis makes me wonder, how many ideas and opportunities have I seen or heard that I immediately dismiss because I think \u0026ldquo;that company shouldn\u0026rsquo;t exist\u0026rdquo;, simply because I don\u0026rsquo;t believe there is a market, or theoretically I can\u0026rsquo;t see it being worthwhile?\nInvestment Funds consistently beating the market - that shouldn\u0026rsquo;t exist. Dominoes selling pizza for $5 when labour costs $15 an hour - that shouldn\u0026rsquo;t exist. A $19B messaging application that\u0026rsquo;s only 5 years old - are you dreaming!!\nI used to see opportunities, but more and more I see problems instead. I want to reverse this trend so that when presented with an idea, I see potential before I see potential problems. This has been an issue for me in recent years, however it has been at the forefront of my mind since the talk from Optiver.\n","permalink":"https://ha.mi.sh/we-shouldnt-exist-but-we-do/","summary":"\u003cp\u003eEarlier this week we had a guest lecturer for our Software Engineering Workshop course (SENG3011 for those at UNSW). The guest was Andrew Maxwell from \u003ca href=\"http://www.optiver.com.au\"\u003eOptiver\u003c/a\u003e, a Dutch trading firm that has a strong Australian presence and invests heavily in IT solutions to facilitate their business (both for algorithmic traders and their human counterparts).\u003c/p\u003e","title":"We  shouldn't exist, but we do"},{"content":"Welcome To DurKode, my personal blog about code, university, and other things of interest. This blog is an attempt to catalogue my journeys and thoughts with relation to nearly anything, and hopefully provide some interesting material for whoever reads this site.\nWho Is The Author (And What Does He Do)? My name is Hamish Durkin, I am currently in my 3rd year of a Software Engineering degree at the University of New South Wales (UNSW) in Sydney Australia. As you can probably guess from the degree, much of my life revolves around software. When I am not studying during the semester, I am working for the companies that sponsor my education (which if you feel like stalking can be found on my LinkedIn profile).\nFrom a technology standpoint, I have varied and mixed interests. While through university I have learnt the programming languages pushed upon us (C, C++, Java), my go to language is Python, especially combined with Django when the task requires it. While I can hack together a front ends for programs, my real interests lie in the work done behind the scenes. Operating System wise, I use mainly Mac (for my laptop) and Ubuntu (desktop) in roughly equal doses, however have used Windows when needed (previous job or for gaming). I feel this information is important to share as it will dictate much of the content of this blog.\nWhy Does This Blog Exist? This blog serves several functions.\nTo store and catalogue my thoughts and ideas. I find that my most productive periods are when I put things down in writing, be they plans, thoughts, a to-do list or anything else. Words are permanent and can be analysed, improved and shared. As amazing as I sometimes (deludedly) consider my brain to be, my thoughts share none of these properties. In essence the primary function of this blog is for my benefit, but hopefully the content will be relevant and engaging for a wider audience. As a tutorial guide. Every so often I will do something technical (aka provision an server OS out of the box) and find myself trying to remember how to repeat the obscure sequence of hacks 6 months down the line. If I feel I will need to remember something again, I will document it here. To improve my writing skills. In all honesty I have done very little proactively to improve my written communication skills in the last 5 years. Given that 5 years is a quarter of my life, this leaves me feeling slightly concerned. I have written next to nothing (with the exception of horrible software engineering reports) since leaving high school. This blog is one step to remedy that. Why Start The Blog Now Before coming to university, I had:\nRun a successful eBay business with my sister Nikki Durkin, who has gone on to found 99 Dresses. Run a profitable proxy network that evaded my schools (and judging by the number of users, thousands of other school\u0026rsquo;s) web filters Sold a script for load testing websites via http proxies (don\u0026rsquo;t ask, I was 13 at the time) Ran shaboogle.com (no longer operational) - a customisable homepage inspired by Google (That unfortunately google cut adSense for) Completed a few websites freelance jobs building websites (in CodeIgniter I mention all of this as the last 2 years have been moreso about enjoying college life rather than working on similar side projects. While that was fun, I feel that now is the time to become more serious about my chosen profession. I want to get back to being more involved with my work, rather than just completing the required courses and doing my internships. I don\u0026rsquo;t want to be an average software engineer. I want to be at the top of my field. If I want to get there, I will have to kick my efforts up a notch, and there is no time like the present to do so.\nUnderlying Technology This blog is powered by Ghost running the Uno theme for Ghost. I chose Ghost as it is a no-frills approach to blogging, allowing me to write my posts in markdown (which makes me happy).\n","permalink":"https://ha.mi.sh/welcome-to-durkode/","summary":"\u003cp\u003eWelcome To DurKode, my personal blog about code, university, and other things of interest. This blog is an attempt to catalogue my journeys and thoughts with relation to nearly anything, and hopefully provide some interesting material for whoever reads this site.\u003c/p\u003e","title":"Welcome To DurKode"}]