Amazon’s Spot Instances – Making Use of Excessive Network Capacity
One of the paradigm changes that has occurred in the past few years is viewing computing resources such as storage space and network capacity in the same way that one views physical resources. Amazon has a tough job managing supply and demand as the aggregate of its users sometimes require more and sometimes less capacity. This leads to situations where it occasionally has an excess of a particular resource. Unlike regular goods however, these are perishables – if no one uses them, they represent a net loss for the company.
This has led to the creation of an entirely new offering known as Spot Instances. Amazon maintains a real-time running pricing of the excess network capacity it has at the moment. Customers place bids on this and whenever the price drops below that did, they get to use the extra resources that Amazon has to offer. Both parties benefit from this arrangement. Amazon gets to profit to some extent from excess capacity that would otherwise be wasted. The customer gets computing power at an extremely low cost. Only those tasks however that are conducive to starting and stopping arbitrarily are a good fit for this kind of system. Applications that require constant uptime and reliability will not be well served. Just another example of innovative cloud-based solutions changing the way we build applications.
The Evolution of Cloud Database Services
It sounds like a fine idea in theory. Using the cloud as a database is very appealing because it is scalable, always available, and cheap. It is possible to build robust applications on such a system. How well does it scale though? Consider a company that has several terabytes of data – and most large enterprises have more than this. The problem of initial data migration has to be solved by storing it in USB sticks and actually mailing it to the cloud service! This is because of the sheer amount of data that needs to be transferred.
This might be acceptable if it was only a one-time operation. But enterprises often have to migrate their data back and forth between locations and this creates an ongoing problem. Add to this, concerns of security and data regulation, and it’s obvious that large-scale database services in the cloud might not be fully ready for prime time yet.
CDN Services and Dynamic Webpages
Content Distribution Networks or CDNs specialize in serving up static data such as media files and pictures to users by storing the content on their servers at locations close to the user. Most websites these days however are composed of dynamic HTML – even simple blog pages will have new comments. This makes it unfeasible to store copies of HTML along with other static content.
CDN services like CloudFlare try and solve this problem by implementing powerful compression methods for dynamic content. Gzip for example is self-referential – but by making use of external dictionaries, it is possible to dramatically reduce the amount of data sent. Other techniques such as caching those parts of webpages that do not change also help to reduce the amount of dynamic data transferred. There’s no easy solution, but CDN companies are making a lot of progress to help speed up the web for all of us.
Is Cloud Computing the Enemy of Open Source Software?
Open source software has a powerful presence in the online computing world. MySQL for example is one of the most popular databases and forms the backbone of a great number of cloud solutions. Unfortunately, the regular GPL under which MySQL is licensed doesn’t require cloud providers to actually deliver any software to their end clients. Enterprises that would otherwise have required a commercial license to sell and distribute products using MySQL no longer need to do so.
This is not the only revenue stream that is being cut off. As large companies become better at delivering cloud services making use of open-source software, they can afford to hire and train their own specialists instead of obtaining them from the creators.
Open source companies are pushing back however with initiatives such as the Affero GPL that covers services delivered over the network. Plus it’s not as if the open source world is in dire straits. The rise of smartphones carrying the Android operating system ensures that the entire base of Linux will remain thriving for a long time to come.