The most impactful lesson I’ve learned in 2019 has to be this quote - “Software Is About Developing Knowledge More Than Writing Code” by Li Hongyi

I specialize in Data Analytics in the cloud and I work with clients from different industries. This year I’ve been leading a data platform implementation on Azure for an insurance company.

One of the key success metrics we defined at the beginning was to increase internal knowledge about the cloud, data engineering, data science. In fact, that was the most important success factor.

Why was it so important? Did the client expect to get thorough documentation on Confluence, or what? Let’s break it down:

Don’t silo people based on their skillset

Data analytics in the cloud is gaining popularity like never before. Cloud services allow developers to benefit from pre-built offerings. The service provider handles the setup, maintenance, security. What took months or even years earlier, now you can implement in weeks.

What’s the downside? There are so many offerings that it’s not clear what to choose. Services are not well integrated. You have to be prepared for breaking changes to upgrade your solution, which might impact everyone in the team.

In our project, we discussed architectural decisions with all team members. It didn’t matter if you were are a data engineer or data scientist.

Tip 1 - Don’t silo people based on their skillset. The more people know about the components and understand the bigger picture, it gets easier for all to make changes (it’s only a matter of time before you’ve to upgrade your design).

Embrace cross-functional teams

Data analytics projects are fun because you work with a group of people with different functional expertise toward a common goal. It allows people to grow in different areas, not only improve their coding abilities.

What’s the downside? Delivering a use case requires many stakeholders on-board: management, business, data privacy, integrations, data scientists, front end, etc. Developers get bored quickly by non-technical discussions - “oh no, GDPR again…“. But it doesn’t mean it’s not important.

Tip 2 - Encourage questions and kill fast the attitude “they don’t get it”. Maybe they don’t get it because you are bad at explaining. Be patient explaining your work, repeat if required, because someone else is patient with you presenting their deliveries.

Explain alternatives and choose the best solution together

If you are an external vendor or consultant, delivering a running system and its code is just half of the job done. But the invaluable knowledge of how it is built and what design choices were made leaves the client’s organization. Even if the system is very well documented, some knowledge is lost every time a new team takes over. Teams change, and eventually, there is no one left who truly understands how it works.

And software is not a static product, but “a living manifestation of the development team’s collective understanding”.

Tip 3 - Instead of presenting “Here is a solution to your problem”, take your time to explain alternatives (pros and cons) and choose the best option together. People will learn new things, feel included in decision making and trust you more.

This article was inspired by Li Hongyi. In his brilliant piece of writing, Li investigates the root causes of bad software. And missing competence is one of the key issues behind bad software.

Another inspiration is a book by Gene Kim, The Unicorn Project. A captivating story about the messy software world.