Why Continuous Deployment is challenging for most companies?
Continuous Deployment (CD) is one of the elements of Lean Startup model and the most controversial one. In simple words, it is an act of rolling software codes to live servers (a.k.a production) as frequently as many times a day. It is an extension of the popular and critical Agile software development practice known as Continuous Integration (CI). In CI, a development team continuously integrates each others work and run a suite of automated test with every change in the code repository (a.k.a. check-in). This allows the team to catch any bad changes as soon as they are introduced so that they can be fixed quickly. Eric Ries took this practice a step further at his company IMVU. At IMVU, they move codes to live servers as many as 50 times a day, a high mark set for others to follow. To be able to pull this off, you would need technical maturity and product management discipline. However, as much as I love to be able to implement it at my company, I see a few issues with extending CI to become CD at most companies.
Review by the product owner. Unless you are a one-man team, you would want your changes reviewed by the product owner before you make it available to the users. In Lean Startup world, most likely the product owner role will be with someone from the customer development team. At least, the product owner is not someone from the product development team. Even the team is co-located, and there is daily interaction between the development team and the product owner, I am not sure how we could avoid the review by the product owner before the changes go out the door. The product owner would want to review the implementation of a feature to make sure it is what he/she asked for. Usually the product owner would have feedbacks for the team. So, I am not sure how bypassing this human review is going to work. The larger the team, the more important the review becomes. If the team is geographically distributed, it is even more important. If you outsourced the development, I don't think you would feel comfortable with changes making to the live servers without you reviewing it.It is not clear how the product owner role fit into this CD at IMVU.
Risk vs reward. Yes, we cannot eliminate all bad changes or bugs through tests. However, I am not sure what incremental value we would get from CD beyond CI. With CD, yes, you may catch some bugs that may still slip through CI sooner (assuming you have a lot of users, which may not be the case for an early stage startup) and it may be easier to track down (there are other ways to reduce debug cycle) the bad codes since only need to work with smaller set of changes. But, at the same time, CD increases business risks (not just ours, but also for the end users), even if we can back out our changes with the flip of a switch. For most business applications, the reward would not outweigh the risk. In fact, it is a good practice to perform deployment during a "scheduled maintenance window" (even if you can do hot deployment) that the users know about so that their is no surprises. However, I can see the reward out weigh the risk for IMVU and similar services.
Non-backward compatible database changes. Database changes are always the most difficult to deal with, and requires more scrutiny and would not be good candidate for CD. In other words, it is difficult to back out (undo) changes if things go wrong. With iterative, incremental development, we encounter these changes more frequently. It is not very clear how IMVU handles these type of changes. It seems from the comments on their blog on CD, they handle the DB changes out of band. That means these are not part of CD.
Although I do not see any significant value in CD to live server beyond CI, I definitely see the value in CD to test/integration servers. In fact, we recently started doing this. It is not exactly CD, more like daily deployment to test servers for the product owner to review and provide feedback. This helps us to be able to deploy changes to live server every iteration (every 2 weeks).
Are you doing CD? Am I missing something?
Review by the product owner. Unless you are a one-man team, you would want your changes reviewed by the product owner before you make it available to the users. In Lean Startup world, most likely the product owner role will be with someone from the customer development team. At least, the product owner is not someone from the product development team. Even the team is co-located, and there is daily interaction between the development team and the product owner, I am not sure how we could avoid the review by the product owner before the changes go out the door. The product owner would want to review the implementation of a feature to make sure it is what he/she asked for. Usually the product owner would have feedbacks for the team. So, I am not sure how bypassing this human review is going to work. The larger the team, the more important the review becomes. If the team is geographically distributed, it is even more important. If you outsourced the development, I don't think you would feel comfortable with changes making to the live servers without you reviewing it.It is not clear how the product owner role fit into this CD at IMVU.
Risk vs reward. Yes, we cannot eliminate all bad changes or bugs through tests. However, I am not sure what incremental value we would get from CD beyond CI. With CD, yes, you may catch some bugs that may still slip through CI sooner (assuming you have a lot of users, which may not be the case for an early stage startup) and it may be easier to track down (there are other ways to reduce debug cycle) the bad codes since only need to work with smaller set of changes. But, at the same time, CD increases business risks (not just ours, but also for the end users), even if we can back out our changes with the flip of a switch. For most business applications, the reward would not outweigh the risk. In fact, it is a good practice to perform deployment during a "scheduled maintenance window" (even if you can do hot deployment) that the users know about so that their is no surprises. However, I can see the reward out weigh the risk for IMVU and similar services.
Non-backward compatible database changes. Database changes are always the most difficult to deal with, and requires more scrutiny and would not be good candidate for CD. In other words, it is difficult to back out (undo) changes if things go wrong. With iterative, incremental development, we encounter these changes more frequently. It is not very clear how IMVU handles these type of changes. It seems from the comments on their blog on CD, they handle the DB changes out of band. That means these are not part of CD.
Although I do not see any significant value in CD to live server beyond CI, I definitely see the value in CD to test/integration servers. In fact, we recently started doing this. It is not exactly CD, more like daily deployment to test servers for the product owner to review and provide feedback. This helps us to be able to deploy changes to live server every iteration (every 2 weeks).
Are you doing CD? Am I missing something?
Loading ...
Recent Comments