The author described how she managed a data science team in her role as VP engineering at a data mining startup.
When you have a team of people working on hard data science problems, the things that work in traditional software don’t always apply. When you are doing research and experiments, the work can be ambiguous, unpredictable, and the results can be hard to measure.
These are the changes that the team implemented in the process:
- improve communication. The different teams (research, development) weren’t speaking the same language. Create a common dictionary
- communicate results in terms of the customer experience and business metrics. Not in algorithm precision terms.
- define what it means to be finished, also for research tasks. Evaluate the results by weighting them with actual customer usage.
- show improvements, show the metrics before and after an experiment. What changed with the experiment?
- communicate why things are hard. Use analogies and examples, get specific.
If you are working in data mining and machine learning research, most of the problems are hard. The thing is, sometimes the reason why they are hard can be solved in other ways — like better/different data, changing requirements, or adding special cases.
- Add deadlines to research. Create a backlog for the experiments, just like a backlog of tasks in traditional software development. These were all the things to try, in priority order. For each of these items, define success criteria, assign them out, and even cost them (how long should this take?).
- schedule monthly demos, where each member of the research team would share their progress. These heartbeat meetings help keep everyone on the same page, and help show the real progress that was being made. These touch points also help create a sense of urgency (since the demos created a deadline) and means to be able to get feedback earlier in the process helping make adjustments to the product along the way.
Communication, shipping regular updates, ask and integrate feedback, keep a prioritised backlog, define when done … all best practices from Agile!