This article has been prepared by Riccardo Tomasi (Head of the IoT Services Management research unit, Pervasive Technologies, ISMB) and Maurizio Spirito (Emerging Trends and Opportunities, ISMB), based on know-how generated by the GreenCom project.
This is a repost of The top 20 Lessons to remember when running an ICT&Energy R&D project with a pilot deployment, originally published on LinkedIn by Riccardo Tomasi (@ric_iot) and Maurizio Spirito (@maurispirito), and submitted to the Europe in My Region 2016 blogging competition.
At the end of April, the GreenCom EU project has finally been completed. The project lasted 42 months and engaged a multi-disciplinary consortium of 7 partners from the ICT and Energy sector operating from different EU countries (Italy, Germany, Denmark, Spain, Ireland). GreenCom aimed at developing a number of innovative CleanTech solutions for various purposes such as: increasing user’s awareness; better exploiting locally-generated PV energy; tapping user’s demand flexibility e.g. by controlling available heat pumps; inter-connecting and monitoring behind-the-meter storage systems, etc.
Unquestionably, the project was extremely ambitious, as it aimed in a single project to define, develop, deploy a good number of prototypical software and hardware components to the field, directly in the homes of 30 real families living in the beautiful Danish Island of Fur.
At ISMB, as coordinating partner, we had a privileged view on the overall experience throughout all the phases: it has been a complex, yet very interesting and rewarding experience.
Looking back, we achieved quite a good amount of outcomes, both in terms of generated assets and experimental results. Nevertheless, we believe that the most important results were the very large number of technical and business-wise lessons and experiences that we learned in the process. It is important to share these with others in the Energy-ICT and IoT research community to enable leverage and acceleration of existing and emerging technologies such as those from GreenCom.
In the following, we made our very personal list of the top 20 lessons that we learned.
20) Keep it holistic: despite being just a R&D project, GreenCom employed a business-driven methodology which “imagined” the potential adoption of each innovation in a number of potential business configurations, looking at the potential interest and value of developed innovations for different stakeholders. As it often happens in exploratory R&D, a potential innovation may be found to be of a limited use or value for a given target customer: this approach to requirement engineering (driven by In-JET with support from EnergyMidt and ISMB’s Innovation Development team) definitely helped the research team in keeping the scope broad, and identifying alternative opportunities for exploitation of their results.
19) Cross-check your concept at international scale: the Smart Grid business is quite complex as significant differences appear in the structure and regulatory framework of different countries. While some innovation may be extremely relevant in some regions, they could be useless or not allowed in others due to diversity in market structure, conditions, customers, etc. It is important to ‘check in’ with experts to ensure alignment of activities at various stages to ensure relevance and compliance. The GreenCom project made a good effort to discuss its concepts and results with experts from various EU regions – receiving very interesting feedbacks and indications. In this work, interactions with SEAI and IERC through partner Tyndall and the collaboration with the Energy@home association through partner ISMB were extremely useful, along with also direct contacts of all partners with several local DSOs and utilities across Europe and abroad.
18) Talk with Pros!: even within the same Distribution System Operators (DSOs) or manufacturer’s organizations, professionals operating different services or developing different lines of products may have very diverse needs and views on what could be considered innovative and useful. This kind of experience is only built with direct hands-on experience: discussing with different professionals from various stakeholders (equipment vendors, DSOs, etc.) have definitely helped inspiring the project development towards what really matters for the target adopters.
17) modular = more sustainable: one result is very clear: the potential value that can be harvested with a single business case in the energy domain (e.g. demand response, energy awareness services, etc.) in many cases will not be enough to pay off alone for the cost of the whole necessary infrastructure. There’s only one way to make innovative energy applications happen: stay modular and follow standards, so that the service you are imagining can share resources and inter-play with others, and leverage common access to the required hardware on the field. In fact the need to comply with standards and regulations can be a major driver in many cases. The ability of bundling and combining hardware, infrastructure and services from different providers in a single eco-system will be a key factor for the sustainability of costs for any Smart Home/Building/Grid solution.
16) Use Open Source and give back whilst a large number of commercial products is available, GreenCom opted for open-source solutions: this provided some very clear benefits in terms of license costs and accesses to documentation and greatly increased the flexibility of the solutions being developed. Moreover, whilst in principle releasing software results as open source is not always mandatory even when adopting open source components (in GreenCom we adopted Apache 2.x-like commercially-friendly licenses), it has an interesting side effect: engaging other interested users is definitely a good way of having your results better known and maybe involve new contributors interested in working with you.
15) Iterative and Agile: a good match: GreenCom has been constructed since the beginning to follow an iterative research approach with yearly development, release and evaluation cycles. When development phases started, we realized that we had a good number of advocates of agile processes among key GreenCom researchers at Fraunhofer FIT, Sensing and Control (SCS) and ISMB. Building upon such common culture, GreenCom partners have decided to embrace even more the agile design and development process principles, resulting in continuous changes and improvements of the system architecture and developed components. This may be a little confusing in the beginning, especially because of the distributed nature of the project, but once the teams got acquainted with the process, the approach definitely paid off. We expect that this will be valid also in other ambitious R&D projects, where, at least in the early stages, requirements are unclear or even contradicting and directions may change progressively as new lessons are learned. Partial mock-ups, demos and prototypes developed in agile fashion also play a crucial role in keeping the discussion with users (see #18) on focus and getting useful feedbacks.
14) Release often & make it easy: while working in agile fashion, it becomes natural to frequently release new versions with improvements and new features to users and the home gateways installed at their premises. Even with just 30 users, the process easily becomes cumbersome and error-prone, if handled manually. Luckily our design was conceived keeping this issue in mind, and after the first two manual upgrades, it became clear that it was worth spending some effort to quickly finalize and enable remote updates.
13) Keep an eye on it, remotely: Troubleshooting systems at user’s premises is not so straightforward if you need to comply with EU’s strict privacy and security regulations. On the other hand if you need to deploy prototypes, it is quite likely that something will go wrong. In the real world there’s only one solution: extensive testing and remote automatic bug reports to be authorized by end-users to ensure that privacy aspects are respected. While these features have been taken into account for future eventual commercial developments, GreenCom was a research project, which softened a little our needs. A dedicated agreement signed by users allowed us to monitor remotely the technical status of the systems in their houses (granted that privacy and security regulations still need to be respected). This has been very helpful in finding solutions in the few, but critical phases where some hardware was not behaving as expected.
12) Prepare to scale, even if you think you don’t need it: back to the days when the project started, we made a preliminary evaluation of the amount and throughput of data that we were expected to generate from the pilot – and the prospects were not so worrying. When we went into deployment, on the other hand, we thought “why aggregating all the raw data we have, if we can just keep it all ?”. In this case, the rationale is clear: once you have spent so much effort to set-up the data collection infrastructure, it’s worth using it to collect as much data as you can – as it could potentially become useful for data-driven research. Luckily enough, the cloud-based Data Warehouse devised by Sensing and Control (SCS) was commercial-grade. Several gigabytes of collected data later with no significant service interruptions or scalability problems, we are grateful for this choice.
11) Keep it simple, keep it valuable: as in many research projects, the initial expectations you have about the needs, the challenges and complexities that you will find are a little distorted by the lack of experience and by the technology-driven mind of researchers and engineers. The main result is that you risk to over-complicate solutions to some problems (that you will not have) and totally neglect to consider other problems (that you did not expect). Sometimes you will need to sacrifice a little the academic relevance of your work, to seek solutions which are simple to use, maintain and do not introduce complexity in your pilot that bring little or no value. In other words: be driven by the application needs not the technology.
10) Don’t believe it until you see it working all together (and for a long time): GreenCom integrated and deployed quite a large amount of Commercial-Off-The-Shelf (COTS) hardware (smart plugs, meter probes, sensors, gateways, heat pump control interfaces, storage systems, etc.). While commercial solutions are certified and overall very stable and reliable, when you integrate them together the result may not always be what you expect for various reasons. For example we decided to use Zigbee-based commercial products where possible due to their use of open source platforms and widespread availability of parts but Zigbee does not always work perfectly in some deployments, particularly where there is a significant distance, extreme radio radio interference (e.g. by ISM-band TV repeaters) and/or thick walls between devices and their gateway (as was found for heat pump deployments in Fur). Despite communication standards that are in place, when digging into details you may run into small (but difficult to spot) interoperability problems, which may appear in the longer run and result e.g. in “lost” wireless devices. The solution: long-running and careful stress-test campaigns, where possible under realistic conditions. Also if possible for wireless devices attempt to characterise as early as possible the RF signal strength under various operating conditions to determine where repeaters may be needed (do not just rely on the technical specifications).
9) Make it fail-safe and self-healing: even if you develop a rock-solid platform over a very reliable network infrastructure, something may still be wrong. Technology is quite reliable nowadays, but it is difficult to stop an uninformed user from disconnecting an internet router in its own house, or a cat from chewing cables. What will your system do in such occurrences? Considering all foreseeable failure scenarios and their impact as part of your technical specification work will definitely help you improve your solution, resulting in resilient components that can still work safely even under degraded or abnormal conditions.
8) Use your Lab, have an Alpha site: when you remotely update software components on a pilot, there is some chance that you may introduce some nasty bug that may cause your remote system to stop operating completely. In order to prevent such issues, in GreenCom we used a mixed approach. New solutions where individually tested in test sites at Tyndall, Fraunhofer FIT, Sensing and Control (SCS) and ISMB premises. This enabled our researchers to be in close proximity to test sites to quickly and efficiently debug the upgrades before migrating to other more remote sites. It should also be remembered that there is significant planning, logistics, travel and time needed and disruption to occupants when a remote site debug is required. Then, we also devised an “alpha” pilot installation, similar to the ones in the main pilot but located in a conveniently close location, so to ‘stress-test’ new updates in a more realistic setting and leaving new updates in full operation for at least 1 full week before rolling them to the pilot.
7) Stay aligned: despite being ambitious, GreenCom is just one of the hundreds of research projects funded by the EC as part of its strategy towards smart and sustainable grids and societies. In an even broader scale, a lot of work is on-going world-wide at research and standardization level. Due to limited resources, of course it’s impossible to keep track of all innovations, commercial products and standards being issued in the domain – but still dedicating some small resources to monitoring and keeping track of recent developments will definitely help the team in moving towards the right direction.
6) Have a Pilot Manager: running a Pilot through a collaborative EU-funded project is not so simple, as the knowledge necessary to run the pilot is literally spread across Europe. Working in such a condition was starting to become difficult for the GreenCom team, until we found a very good organizational solution: nominating a Pilot Manager, in charge of keeping the team synchronized on all pilot-related issues, located (relatively) close to the deployment site in Fur and familiar with the set up and people, helping everybody in quickly finding solutions and talking with pilot participants to explain and prevent issues. This has not been easy, but EnergyMidt team really did an excellent job on Pilot Management, also thanks to the help of a few GreenCom technical components, such as the Network Monitoring and Control framework raising alarms in case of device failures and tracking pilot issues.
5) Keep the team engaged: EU projects such as GreenCom are clearly collaborative in nature – and typically see results of different partners being integrated and deployed. Keeping the technical vision of teams from different companies is not always straightforward at all times. While agile approaches help keeping the team together, you really need to put in place a strategy for effective and frequent communication among researchers. Mailing lists and trackers help, but they are a little impersonal. In GreenCom we introduced short, weekly stand-up conference calls and a common SCRUM board: this really helped the team in identifying issues in advance and quickly finding solutions together.
4) Pilot location is important, Pilot participants will make the difference: As with any project dealing with low-voltage grid and consumer issues, results naturally vary depending on consumption patterns, amount of PVs and storage systems install, etc., thus making the choice of the location of the pilot a very important factor. On the other hand, there’s a factor which is even more important: your end-users. GreenCom was very fortunate on this aspects, having secured the collaboration of the families of the Innovation Fur project. Users feedbacks, their patience and availability in helping you developing and troubleshooting our components has been a key success factor for GreenCom.
3) Get in contact with other pilot projects: living labs and pilot projects provide a unique environment for developing R&I projects, giving the possibility to directly try in the field innovations that would otherwise remain at paper or proof-of-concept level for years. Although rewarding, the competences for deploying and operating a pilot require a comprehensive set of operational, planning and flexibility skills, which are not typical in R&D teams and not even in operational teams, which are typically used to work with mature, fault-tolerant technologies. A solution which can really help in mitigating the problem is to get in contact with projects developing similar experiences. In the case of GreenCom, the insight suggested by initiatives that developed large-scale pilot experience such as Energy@home and EcoGrid were definitely useful in preventing known issues. Of course it is important that this is a ‘give and take’ exercise where we are happy to share GreenCom experiences with other pilot projects and everyone benefits.
2) Check and validate your data: in a project such as GreenCom, running hundreds of low-cost embedded sensors and devices distributed at users’ premises, it is likely to bump into defective devices, gateways or devices being disconnected, prototypes failing, installation errors, etc. All these likely events impact the quality of the flow of data, and thus any planned use of the data. Checking high-level metrics (e.g. devices liveliness, amounts of data samples collected, etc.) is not enough and actually manually checking all data flows one by one is not practical with such a large number of devices. The solution? Review your data using a mix of automatic and assisted procedures. Developers that rely on your data will thank you for the quality of your data sets.
1) You are at the Front – deal with it: regardless of all the strategic Smart Grid directions and evolutions pushing forward more and more synergies and collaboration between the ICT and electricity grid domains, experts with background in electricity grids and power systems still have a very different mind-set, jargon and drivers in mind, compared to professionals working on cloud services, embedded IoT systems and internet-oriented technologies. Any multi-disciplinary project working at the front and around the boundaries among these two worlds must expect to run into clashes of plain terminology, culture and vision at some point, or even just communication problems. GreenCom, being executed by 5 companies with strong ICT competences and just 2 partners with strong expertise on energy and grid-related issues had its challenges on these aspects. There’s no easy way to solve this, but things can get smoother if participants pay attention on carefully explaining terminology and goals when communicating across the two worlds, as well as explaining background and drivers for key decision, even if sometimes it may sound obvious for experts in your own domain.
This was just a short selection of the most generic (but important) lessons that we learned in the project and shared in the hope that they will be interesting or useful to other similar projects in the future. Naturally the project has developed a much wider number of findings: this is just the tip of the iceberg.
If you are interested in knowing more, feel free to get in touch with us!
This article has been prepared by Riccardo Tomasi (Head of the IoT Services Management research unit, Pervasive Technologies, ISMB) and Maurizio Spirito (Emerging Trends and Opportunities, ISMB), based on know-how generated by the GreenCom project. Thanks to Michael Hayes (Senior Program Manager ICT for Energy Efficiency at Tyndall National Institute) for the precious suggestions and corrections. The picture in the headline has been taken on the beautiful Island of Fur, where the GreenCom pilot is located.