Overcoming the Cold Start Problem: How We Make Intractable Tasks Tractable

by Azin Asgarian and Franziska Kirschner

I woke up this morning (somewhat unsurprisingly). After poking my head out of the covers and feeling a cold chill in the air, I made the executive decision not to proceed with this course of action and promptly withdrew back under the covers. Ten minutes later, my alarm decides it is time to snooze no more and kindly invites me to follow suit. Begrudgingly I got up; another winter morning plagued by the Cold Start Problem.

In the same way owners look like their dogs, Machine Learning Scientists think like their AI. At Tractable, our AI for accident and disaster recovery suffers from the Cold Start Problem too, albeit all year round. Imagine you have an AI that is, say, really good at spotting scratches on Toyota Priuses, because for some reason you chanced upon a large, labelled dataset of Toyota Priuses that may or may not be damaged. Once the AI has trained by looking at all these images, it becomes a pro at knowing when a Prius is damaged and when it’s not.

Now, imagine you want to scale your AI to detect scratches on Lamborghini Aventadors instead. But you don’t have a large, labelled dataset of Lamborghini Aventadors. Your AI can’t really make an accurate judgement, simply because it doesn’t know what a Lamborghini Aventador is meant to look like, scratched or not. The AI, like the researcher who trained it, has trouble getting going when it’s cold, and needs to warm up to function smoothly.

Tractable’s AI is used for visual car damage appraisal in thirteen countries over three continents. Every time we enter a new country,

1. the cars look different and
2. the way those cars are repaired is also different.

We often start cold. An AI trained to write repair estimates à la Japanese Adjuster is unlikely to fare well writing a repair estimate in Poland. With so many cold starts, we needed an AI warm-up routine to help us adapt quickly to new countries.

In Machine Learning, (1) and (2) are types of domain shift. (1) is an input shift – meaning the inputs—in our case, images—our AI sees look different. This is because car models differ around the world (think about our Prius vs. the Aventador).

(2) is a conditional shift – meaning, the repair decisions—which are conditional on the damage to the car—are different around the world. To put that another way, if you have the exact same accident in Thailand, the UK and Poland, the repairs will be different in each country. We’ve got 99 problems and the solution to each depends on where on Earth you’re standing.

Tractable is working with the R&D Team at Georgian to expand our ability to handle domain shift. We’re using research from an area known as domain adaptation, which looks at how we can use information from other domains to learn about a new domain. There are several ways to handle a domain shift problem, including:

1. Ensemble learning – the AI equivalent of getting a group of experts in a room, and putting their heads together to try and solve a difficult problem, and
2. Instance- and parameter-based transfer methods – a whole host of techniques designed to understand the abstract similarities and differences between domains, so that we can control how much our AI can generalize or specialize.

Through a combination of ensemble learning and domain adaptation techniques,  Tractable can leverage its proprietary datasets to the fullest and expand its AI models to new markets quickly and efficiently.  Our global AI can achieve a 20%+ performance boost over non-adapted models. Perhaps I’ll set my thermostat to switch on the heating before I wake up tomorrow. Starting warm seems to work well for AI, so maybe it’ll work well for me too. 

If you want to learn more, you can watch our talk on how we did this from the TMLS Conference here!