After ten years of selling embedded software tools and services to a wide range of semiconductor and systems companies, I thought I'd share some observations as follows:
- A tiny minority of companies have a performance problem that they have recognized and quantified.
- Most companies don't do a good job of measuring the performance of their products.
- Agile development methods, for all the good things that they bring, have made the performance issue worse.
- Performance problems are usually the cause of products which are too slow; but products consuming too much power or using too costly components head there too.
- Almost all companies have an opportunity to improve market competitiveness, product differentiation and operating margins by optimizing software performance.
When we started off in software optimization at CriticalBlue, we naively imagined that many customers would have performance challenges with which they might need help. However, after a couple of dozen interesting meetings that ended with statements along the lines of “Thanks. If we have a performance problem, we certainly know who to call,” we realized that we must be missing something.
Many successful projects later, we think we understand how this works. Almost no-one has a performance problem. Of course they do, but either they won't admit it to themselves or else they simply don't know that it exists. It's sad, but true, that oftentimes companies don't know that they have a performance problem until their customers tell them, by which time all manner of ordure is hitting the air conditioning device and a mad panic ensues.
You'd think that there must be another way, and there is — optimization of embedded software performance can be used as a business differentiator and a game-changer. Imagine being able to extend the life of an existing product, or even release a new product, based on unchanged hardware and optimized software, delivering more capacity and throughput. Such a new product could leapfrog the competition, afford a premium price, and — since it is based on existing hardware with a new software build on top — brings nothing but profit to the business. It's a nice story and it's also a true one.
Traditionally, performance tuning has been an afterthought. Create the behavior you want and then optimize at the end of the development, often in parallel with functional testing. In this scenario, there is almost always time for some performance work, but never enough. Agile methods are all the rage at present and, for all the benefits that this approach brings (and there are many), Agile makes the performance challenges I have been describing worse for the simple reason that short sprints are typically laser-focused on delivering demonstrable functionality. This means that there is just no space in the schedule for fine-tuning. As a consequence, the whole area of performance measuring, monitoring, and mediating steadily deteriorates.
So how can this performance issue be turned from a demon into a bluebird, or rather from a hidden evil that will bite you in the nether regions into an opportunity for you to release a more competitive product? The answer is “on an ongoing basis” and not just at the project/sprint end — to assess your software at multiple levels and to establish how much better it could be and how much effort it would take to capture that benefit. Once this is done, you can grade the results and look at the return on investment (ROI) of the various pieces of the project. From there, you can cherry-pick what makes the most sense and implement the appropriate optimizations. Assuming that your initial analysis was accurate, you'll soon be unveiling an exciting new offering, and it's likely that the development time will have been a lot shorter than for your previous product!
Opportunities to optimize will exist at different levels of your software stack and at different places where hardware and software meet. Some examples of where to look are as follows:
- Compute-intensive library optimization towards underlying hardware architecture.
- Data structure re-factoring to optimize use of underlying cache architecture.
- Dynamic memory footprint optimization.
- Compiled code and JITted code optimization.
- Algorithmic choice and partitioning.
- Thread organization and management .
These are the areas where we are most likely to work directly with customers. In fact, this kind of work is highly specialized and many development teams don't have the expertise to do it by themselves. Also, much like verification, there is a strong argument that perhaps this should actually be performed by people other than the original developers themselves. This is the vast majority of the work we do at CriticalBlue. We are your optimization partners, operating alongside your development team, monitoring software performance, and objectively searching for places where code can be tuned. A subset of our expertise and experience spans OSes such as Linux, Android, iOS, and bare metal; development environments such as gcc, LLVM, Jenkins and Bamboo; and vertical markets such as mobile, telecom, payments systems, vision systems and cloud products.
With today's embedded systems having ever-decreasing development windows, and software driven products being rushed to market in growing volume, it's more important than ever to get back to real engineering work, which must include performance optimization. We must embed optimization into our Agile processes to do a good job initially, and then rely on new capabilities such as over-the-air software updates to maintain competitiveness and extend product lifetime. In other words, we need to take care of performance throughout a product's operational lifetime, not just its development lifetime.
Fundamentally, we like to think of ourselves as the guys who make sure you never have a performance problem and help you create a product whose specification surprises you and your competition. So, is there a premium product bursting to get out of your existing hardware and software? If there is, how much would it be worth to set it free and place it in front of your existing customers as well as your competitor's customers?