Software Development Trends

download Software Development Trends

of 27

  • date post

  • Category


  • view

  • download


Embed Size (px)


Talk given to the Software Engineering professional community in BT on 21 October 2009.

Transcript of Software Development Trends

Trends inSoftware DevelopmentKerry Buckley


Open Source






Document DBs



First a disclaimer: this talk is based on my personal impressions of software development trends, so dont treat it as company or community policy.

I certainly dont claim to be an expert on any of this, and a lot of it is just gleaned from blogs and twitter, so will reflect the particular echo chamber I happen to inhabit (mostly the Ruby, Agile and Software Craftsmanship communities).

Im not in charge of anything either Im just a developerand just to prove it, heres my directory entry.Ill start with the most obvious thing people probably think of when they consider development trends languages.

The general consensus seems to be that there isnt going to be a single Next Big Language in the same way as we had FORTRAN, C, Java etc. The buzzword these days is polyglotism, meaning that people are proficient in multiple languages and use the one most suitable for the job (maybe DSLs and dynamic languages at the top of the stack, and functional languages lower down for reliability and scaling).One of the things thats made mixing and matching multiple languages much easier has been the growth in the number of languages that can run on common platforms alongside the usual Java and C#.

These all run on the Java platform, and the storys similar on .NET.Unless youve lived in a cave for the past few years, its hard to have missed the growth of dynamic languages like Ruby and Python. This graph shows that demand for these is still way below that for Java, but it gets more interesting if you look at the graphs for growthYou can see here that Ruby in particular appears to be growing at a fast rate, presumably driven by the use of Rails for web development. But theres another language thats growing even fasterI searched for the term Ajax instead of Javascript because thats the specific use case thats driving the resurgence of Javascript, but you can see from the graph that demand has rocketed.

Ajax had a bad reputation for years, but libraries like JQuery have removed most of the pain from using it, and allow dynamic behaviour to be added to a page in a structured unobtrusive way.As well as dynamic languages, another category thats getting a lot of interest lately is functional languages, including for example Erlang, Haskell, OCaml, Clojure and F#.

Because they avoid state and side-effects, functional languages are well-suited for massively parallel execution, which is becoming more important with the rise of multi-core processors and cloud computing.

As well as these production applications, many people are learning functional languages to give them a different viewpoint, which can add another tool to your collection even when working on OO languages.Outside the enterprise world, use of open source software is almost universal, but even in large companies adoption is starting to become much more widespread (or at least more officially-approved). These are just a sample of the open source tools and products already in use in BT.

Consumption is only one side of the coin though, and businesses are starting to realise the benefits of contributing to open source projects, and opening up the source of their own software. This allows collaboration and sharing of effort on common tools, leaving more capacity to concentrate development on systems and products that are core to the companys business.Contributing to open source projects has become much easier thanks to distributed version control, and particularly sites like GitHub and BitBucket.

Unlike traditional source control systems like Subversion, distributed tools like Git, Mercurial and Bazaar mean that each developer has a clone of the entire source repository, and can pull changes from anyone else. This means you can work on a fix or enhancement, control your changes locally, merge in concurrent upstream changes then either submit a patch or request that the project maintainer pulls the commits from your copy of the repository.

DVCSs also allow you to easily maintain a fork of a project for example to add support for some local requirements without losing track of changes to the parent project. This screenshot shows part of the network of repositories for a single project each coloured line represents one users copy, and the arrows show changes being merged from one to another.

Obviously in BT were standardising on Subversion for various reasons some of which are no doubt perfectly valid ;-) but its worth being aware of tools like Git anyway, as they are rapidly becoming the norm in the open source world, and have lots of advantages.A few words about software architecture. Dont forget that as I said earlier Im just a developer if anyone tries to call me an architect therell be trouble!

From the stating the bleedin obvious department, the web (with whichever version number you choose to append to that word) continues to be central to an awful lot of software architectures, both within and outside the company.

As well as the human web, this also includes web services, which are increasingly moving away from SOAP towards REST (or at least simple data over HTTP).Another buzzword that no-one can have avoided is cloud, which again means different things to different people.One thing thats certainly already a reality is the growth of web-based services that can handle everything from hosting your application to source control, continuous integration and fault tracking. These allow small companies or independent developers to release staggeringly quickly with minimal startup cost, and to scale successful projects relatively easily.Many modern web application frameworks are designed to scale out by adding more servers, rather than scale up by running a single instance on a massive machine. This is well-suited to deployment to commodity hardware or cloud compute instances, but distributing the database layer can be difficult with traditional relational databasesThe problem of distributing relational databases is one of the reasons why document-based or key-value databases are becoming more popular. Another is the tendency of frameworks like Rails to treat the database as simply somewhere to persist objects, with much of the logic that traditionally lived in the database moving up to the application layer.

Document databases store relatively unstructured data in an ad hoc fashion as simple key-value pairs, a bit like a giant persistent hash. They dont have schemas, and dont use SQL for querying (for example CouchDB uses a Javascript view model, exposed using a RESTful JSON API.Obviously agile development methodologies continue to spread, but as adoption grows, particularly in large organisations, theres a concern that the original intention is being watered down, with too much concentration on process at the expense of technical excellence and delivery of value.Most large organisations that adopt agile development start with Scrum, which provides a good framework to get started from, with its emphasis on iteration and lightweight project management. Unfortunately Scrum doesnt tell you what to do within those iterations to actually develop quality software, and thats where the trouble startsIf you start trying to develop software in two-week iterations without also having the right technical practices, its going to end in tears. Scrum explicitly doesnt cover these, and Scrum experts generally recommend following the engineering practices from Extreme Programming within the management framework of Scrum.Some of the key XP practices are automated testing, pair programming, continuous integration and refactoring, and were starting to see many of these pushed more strongly inside BT with the work that DSO are doing on CIT.

In certain parts of the wider world (such as much of the Ruby and Rails community) practices such as test-driven development have become the norm rather than the exception, but in general adoption still seems to be relatively low.While continuous integration is becoming more of a baseline practice, some teams are taking it a step further to continuous deployment, where changes are deployed almost as soon as the CI tests have completed. Flickr is a well-known example, with changes deployed multiple times in a day.Ideas from the Lean world are also starting to be applied more widely in software development, particularly in terms of reducing waste and limiting work in progress. Some teams are starting to apply a Kanban approach to user story flow, with continuous delivery instead of fixed iterations.Many of the original authors of the Agile Manifesto are now trying to promote a shift back to those original values, away from a watered-down, corporate-friendly version of Agile.

One example is Brian Maricks ARTA, which stands for artisanal retro-futurism crossed with team-scale anarcho-syndicalism. The aims are similar to those of the original agile movement, but the name was chosen deliberately to stop people being able to claim oh, we already do that without at least stopping to read about what it actually means. Theres more detail on related trend is the emerging Software Craftsmanship movement. This is