JATO are one of the largest global suppliers of automotive data. JATO employs a worldwide team of researchers to capture vehicle data, which is then combined with other public and private feed such as registrations and sales data. From this information, custom data sets are built for specific needs. Automotive manufacturers may choose to take feeds that allow them to compare themselves with competitors in different regions, while fleet managers might take total cost of ownership (TCO) data.
JATO have been collecting this data for over 30 years, and a key challenge was the complexity, and volume of data being processed. Over time and between markets, the quality and breadth of the data captured varies greatly. This posed many problems when building the final data sets.
Traditionally, JATO have used a mixture of in house technologies based around C, C++, VB. .NET and SQL Server, and their longer-term strategy was a new technology platform based on the JVM platform, but an early version of this experienced performance problems.
Over an extended period, Can Factory worked closely with UX experts at a design agency and with JATO’s architecture team to refine user requirements and define the new platform. The emphasis was on understanding the full “end-to-end” data flow. Tasks were roughly split into two streams. A “proof of concept” team worked with the UX and domain experts to develop a prototype version of a key service. This had a rich browser based UI developed using Ruby, SASS and the backbone.js framework, connecting to a Java/Groovy backend.
BDD techniques were used to capture and refine domain rules. A “data pipeline” team looked at the wider data modelling, data cleansing and data distribution problems. This process was managed through a number of facilitated workshops, technical investigations and technical prototypes. A particular challenge was that while the problem has many aspects of what could be considered Big Data/ NoSQL, the deployed solution had to be technically conservative.