Project Overview
In this project's scope, I worked on the Connected Service feature, which is intricately designed to collate extensive sensor data from a fleet of vehicles. This data, derived from an array of sensors strategically positioned throughout the cars, particularly near critical components like the wheels, is meticulously captured and stored within the robust infrastructure of AWS S3 and S3 Glacier storage systems.
The team's primary mandate was to enable the analytics teams to unlock the value hidden within this sensor data, transforming it into actionable insights that could drive operational improvements. A notable application was developing a sophisticated analytics solution to detect potholes in specific geographic regions—a task of significant value for municipal maintenance and driver safety. To achieve this, I spearheaded the development of precise GraphQL payloads. These were tailored to query and extract the relevant sensor data, focusing on the nuanced vibrations and other telltale indicators captured by the sensors near the vehicle's wheels. These payloads initiated a seamless data transfer upon activation, channeling it into a secondary S3 bucket allocated for further processing.
In the subsequent phase, the team meticulously engaged in post-processing this data. This involved applying advanced analytical and big data techniques to distill the raw data into coherent insights, which we then formatted into client-ready reports. In parallel with these technical endeavors, I undertook a strategic review of the data management practices. Through a detailed analysis of data usage patterns and storage efficiency, I identified opportunities to optimize costs. By configuring the lifecycle policies of the S3 buckets, I reduced unnecessary data retention and transfer costs, thus achieving a more cost-effective storage solution without compromising on data accessibility or integrity.