Please update your browser.by Tara Paider
January 1 — 7 min read
One of the goals from the start of our journey was to make sure our 600 legacy ETL engineers could adapt and grow with us as we underwent our transformation. We did not want to replace the existing developers but were instead determined to invest in their development so they could modernize their skillsets but also so we could retain their deep knowledge and expertise.
It was a tall order – these developers were moving from using drag and drop user interface (UI) capabilities to being open source, full stack developers. We had a large global team that required both standard Java and Spark training, as well as extensive hands-on training to learn the in-house DPL framework and standards. We assessed several global external training vendors to provide Java and Spark education, and we also found a few passionate internal developers to begin developing DPL training materials.
It was imperative that we move quickly in delivering this training at scale to jumpstart our modernization initiative. Within just a few months, pilot training contents were available across the U.S. and India. Most of the engineers were thrilled to have the first intensive technical training they had seen in many years.
What was in it for them?
The firm paid for their training and gave them plenty of time to complete the training and catch up on the learning curve. It was a large investment in their future but we knew that recruiting, hiring and onboarding resources to replace these legacy experts would have been a much larger investment.
We did, however, need to hire new resources – we had to increase headcount by more than 100 new developers and sought experienced Java and full-stack engineers to seed the existing teams. We were very successful recruiting from emerging tech programs such as local boot camps, which are increasing in popularity across most regions. We also saw an amazing camaraderie between the seasoned ETL developers and the new hires. They were eager to teach each other for the benefit of the overall transformation.
Classroom or online training only goes so far, though. If the developers did not have the immediate opportunity to use their new skills, they would be quickly forgotten. So, we created adoption parties – one-day sprint sessions led by agile coaches and data pipeline experts, where teams brought their own production ingestion data flows and worked together to build ingestion pipelines using DPL. Teams could participate in several adoption parties, jump-starting their own modernization scope and ensuring the skills they learned in the classroom were complemented with hands-on training and learning across their peer group.
To supplement our engineers’ training, we also created our own internal support network. Consisting of internal chat channels, stack overflow inventories and assigned "resident experts" from inside our own teams, we created multiple communication for developers looking for help on features or code issues. By sourcing our own help lines, we decreased the burden on the centralized DPL teams, allowing them to focus more on building new features. Everyone who volunteered as monthly support leads also got a boost to their DPL capabilities because they were actively troubleshooting Java and Spark issues.