Not long ago the folks over at DotNetRocks aired a podcast on application testing in the Cloud. In this podcast, they talked about the benefits of testing your code in the Cloud across a range of applications including web, mobile, and client/server.
While the podcast covered a spattering of topics related to testing applications in the Cloud versus using on premise infrastructure, it did not really address the reasons why you might choose the cloud. What I want to know is, what drives developers and product teams to move their development and testing to the cloud? Sure there are the obvious Cloud benefits: reduce some infrastructure costs, scale easier, and work across the globe. But I’m more interested in what the implications are for the application lifecycle and ultimately its success.
Defining Development & Testing in the Cloud
As with all things novel and technological, it’s easy to get confused just by terminology so let’s define a few things. First, “The Cloud”: I’m referring to public Cloud service providers. I don’t mean your internal Cloud, and I definitely do not mean the hypervisor on your local machine. So next, what do I mean by “Development & Testing”?
There is just Development, and there is just Testing. But “Development & Testing” is the process of taking what has been coded, and making sure it works across commonly used workloads. Some call this Application Lifecycle Management (ALM) minus the release process. It’s important to note that I’m not referring to application testing. Application testing is taking new versions or completely new line-of-business application, playing around with their features, or making sure adding an update does not break things. On the other hand, Development & Testing relates to the application you are building for an internal or public audience.
When I talk about Development & Testing in the Cloud, I’m referring to moving the bulk of the infrastructure used for Development & Testing to the public Cloud. IaaS and sometimes PaaS would definitely fall into this category. All applications — even mobile applications — have a server somewhere they are referencing, and this server is likely crucial to the application. Some applications also use Cloud services for worker threads or like databases. These may or may not be included in Development & Test as many of these services once set do not need to be touched, and can be consumed both by Development and Production.
The 5 types of activities that happen during Development and Testing include:
- Automated Testing
- Manual Testing
5+N are the staging to deployment and release portions of the development processes, and they live in the land of Production. At this point you are asking yourself, is there a difference between Production Infrastructure, and Testing Infrastructure? YES!!
The Difference Between Production Infrastructure & Testing Infrastructure
Production infrastructure is similar to the difference between having an agile Ferrari, or a ¾ ton truck with snorkel and smoke stacks. Both are going to get your buddies to think you are really cool. But each are cool for different reasons. The Ferrari is cool because it can weave in and out of traffic, and move really fast. The truck is cool because it’s like a rock, stable and reliable. You take the Ferrari out to see what it can do, but you put the truck to work.
The same is true in the Cloud. Your Development & Test Cloud has to be flexible, and orchestrate quickly. Your Production Cloud, however, is something that you set up one time and use on a daily basis You don’t need to change the configuration at all; you just use it.
In technical terms, this means for a Dev/Test Cloud you need to be able to:
- Snapshot robust environments
- Replicate environments on demand
- Collaborate and share environments
- Replicate configurations without manual effort
- Version configurations
- Integration with development tools
Whereas in a Production Cloud you need:
- The best machine-level SLA you can find
- Lots of resources
There are public Cloud providers that are tailored for one of these scenarios over the other. For example CloudShare is tailored for Development & Testing, whereas Azure and AWS are built for Production. In CloudShare you can snapshot multi-machine environments and maintain network and memory state where you last left it, whereas inAzure and AWS you can snapshot the disk of individual machines, but you will have to manually reconfigure your environments if you ever revert.
It’s these efforts in the Development and Testing cycle that will kill you.
Back to the development activities list.
Faster Dev/Test Cycles Depend on Faster Access to Infrastructure
Most developers are not going to allow their IDE to leave their local machine. To take it one step further, most developers have their tricked out Wall-E themed — if they even allow you to see it, and its intense power to produce amazing code — control-laden custom Visual Studio.
Unless you are trying a new control from your favorite component vendor, you probably won’t use Visual Studio in a VM in the Cloud.
Once a developer produces that amazing code, they have to build it. If you are an on-premer, you are pushing your code to that virtual machine your nice IT guy provided for you. Problem is, the chances of the IT guy getting the configuration correct are low. Let’s hope it’s not a multi-tier application because this is going to be way too complex. The chances of the IT guy even getting you that VM in less than one hour, or even in a timely fashion, is also very low. This means you and your team are twiddling your thumbs in frustration waiting to test your new build.
Once you build your code on some infrastructure somewhere, you are ready to do some automated or manual testing. This is where the on-premer’s start sweating the number of tests and configurations they have to run. Because, again they have to go to their IT guy and ask for the infrastructure to run the tests. Usually what happens is that they end up limiting their tests to only the essential ones because of time, or limitations in the number of variations and size of infrastructure they can test on.
If all goes well, and you have a satisfactory build you will move it to staging, then to Production. This process should be fairly established and unchanging.
So you might have noticed a trend that, if you’re a developer, you might not be terribly aware of because it’s so common to you. The infrastructure used for Development and Testing is highly volatile. Meaning it goes, up, down, left, right very quickly. When you try to fit this into an on-prem lifecycle, you are immediately limiting yourself.
And this is just for workloads that you MUST run. What about those workloads that would be NICE to run? All the things you dream about doing, but you can’t because it’s too prohibitive, and risky. The opportunity cost of running on-prem is huge. It limits the Development and QA team’s ability to run tests, or even experiment.
The reason modern applications are moving so quickly, and evolving to satisfy users’ needs more rapidly is because the modern development team experiments. They get to test new things, fail fast, learn, and move on. All without a meeting, purchase order, another meeting, slap on the wrist, another meeting, and finally a rejection.
You can’t grow your application’s capabilities if you are not innovating and testing new things. This might be new configurations that make the application run three times faster. Or it might be brand spanking new APIs that, if successful, will be that one killer feature that will make you competitive, or make your users swoon for your application.
If you use the Cloud for testing, you can do anything. Because in a Cloud built specifically for Development and Testing, when you break something, you giggle a little, revert from a snapshot, and test again. No IT, no hiding from your boss, no forehead slaps (well unless you did something really silly).
Why You Should Move Your Development & Testing to the Cloud
So what are the real business drivers for moving your Development and Testing to the Cloud?
- IT and Development teams become friends
- You can test more scenarios without additional effort
- The success of your application increases exponentially
In most organizations, even ISVs in Silicon Valley, IT and Development are not the best of friends. While to most outsiders they are one in the same, or very closely tied. When you are part of these teams you know that developers get mad at IT for being slow and not caring enough about what they need. And IT gets mad at developers for taking up Production resources, asking for weird things like specific ports being open and bizarre configurations, and finally distracting them from their day job. Because the Dev user is not the typical user, every interaction between the two is unique and usually frustrating.
By moving Development and Testing to the Cloud, this frustration goes away. Developers get off IT’s back, and what used to be “encounters” in the break room, are now much more friendly conversations about the latest MMORPG.
When you move your testing to the Cloud you can do very cool things like make small variations in tests, or double the amount of tests you run without additional effort. You can expand your tests to all run on their own isolated infrastructure to make a true test without managing as many strange variables as possible. We all know that machine-level configurations are the most adversarial application issues that can arise. But in the Cloud, you can run separate tests on configuration identical environments without any effort. Not to mention, you can do bursting, and load testing that you never thought possible with your on-prem infrastructure.
And finally, here is the kicker. On-prem will limit you, and your application, from becoming the modern cutting edge bundle of code you imagined it to be. If you want your application to be modern, even if it is a legacy database application aspiring to have a mobile and web interface, you have no choice but to move to the Cloud. Because, that cool server under your desk is neat, but it’s holding you back. Turn it into a gaming server, and move your day job to the Cloud.
If you are an on-premer, stop it. You love your application, and you want it to be successful and modern like all the other cool kids. So move your Development and Testing to the Cloud now! If you are already in love with the Cloud, and moving your Development and Testing there. Remember that not all Clouds are created equal, and what you are looking for is that special functionality that allows you to share, snapshot, revert whole multi-machine environments in their exact state on the fly.
My name is Chris Riley, and I too used to be a server hugger. If I can do it, then you can too