Connection Holdings, LLC, a leader in performance marketing, required a robust solution to manage and optimize its extensive data operations. Vested Exchange was tasked to streamline the collection, processing, and presentation of data from various third-party sources. The primary goal was to enhance their ability to generate accurate, real-time insights that would drive better decision-making and improve overall operational efficiency. In time the goal was to also use AI and ML models to conduct pricing, financial and accounting analysis.
Connection Holdings, LLC is a prominent player in the performance marketing industry, specializing in lead generation for various sectors including Medicare, solar, home services, insurance, and debt consolidation. With over 150 million leads in their database, they cater primarily to call centers and companies operating in these verticals. Their operations are data-intensive, requiring sophisticated tools to manage inventory, ensure compliance, and optimize pricing and performance.
Before the implementation of the Vested Exchange reporting dashboard, Connection Holdings faced several critical challenges:
With over 150 million records in its database, Connection Holdings needed an efficient way to manage this vast amount of data. The sheer size of the dataset made it challenging to quickly and easily access relevant information. Specifically, there was a pressing need for a system to break down their inventory by state or vertical. This granular view was essential for managing their operations effectively across different regions and market segments.
Given the volume of data flowing through the system, ensuring compliance and the ability to efficiently audit this data was a significant challenge. Connection Holdings operates in heavily regulated industries like Medicare and insurance, where non-compliance can lead to severe penalties. The existing processes were manual, time-consuming, and prone to errors, increasing the risk of non-compliance. A robust solution was needed to automate and streamline compliance checks and auditing processes.
Another challenge was the need to set up effective pricing and performance metrics based on the data. With so much data being processed daily, Connection Holdings required a system that could accurately specify the correct metrics and pricing each time data was exported or used. This capability was crucial for optimizing revenue and ensuring that pricing strategies aligned with performance outcomes.
Different parts of Connection Holdings' business required visibility into different metrics. For example, the supply side of the business needed to focus on inventory and sourcing metrics, while the delivery or sales side required insights into sales performance and customer acquisition metrics. The existing reporting system lacked the flexibility to provide customized views tailored to the needs of each business function. This limitation hindered the ability of various teams to access the specific data they needed to make informed decisions.
To address the challenges faced by Connection Holdings, we developed a customized reporting dashboard for Vested Exchange, integrating advanced data management and reporting capabilities. The solution involved the following key components:
The solution integrates data from various third-party platforms through a sophisticated data pipeline. We are currently migrating data processes to utilize API calls via AWS Glue, which stores data in a DynamoDB database within AWS. This data is then streamed from DynamoDB into Snowflake via Estuary, ensuring efficient data management and real-time availability. For specific use cases, where data is updated less frequently, we also utilize Amazon S3, allowing for daily or weekly data pushes. This dual approach optimizes both performance and cost by balancing real-time updates with batch processing.
The dashboard is designed to handle large volumes of data efficiently. For platforms that support CSV exports of data over a specified number of days, we minimize API calls by using a single API call to export the data and a follow-up call to retrieve the file. This approach reduces operational costs compared to continuously running DynamoDB processes, which are set to update data every 15 minutes. This flexibility in data collection methods ensures that the system remains cost-effective while maintaining up-to-date data.
Once the data is collected, it undergoes a rigorous normalization process to standardize information from various sources. The system also includes a user interface (UI) that allows users to correct any data anomalies through a feedback loop. This ensures that all data is accurate and consistent before it is appended to the Snowflake database, where it can be utilized for reporting and analysis.
The dashboard includes several critical features tailored to Connection Holdings' needs:
The dashboard provides customized reporting capabilities to meet the distinct needs of various business functions within Connection Holdings. For example, the supply side of the business can access metrics related to inventory and sourcing, while the sales or delivery side can focus on sales performance and customer acquisition metrics. This customization ensures that each team has the visibility they need to make informed decisions, tailored to their specific responsibilities.
To ensure that Connection Holdings always has access to the most current information, the dashboard supports real-time data updates. Data collected and processed through DynamoDB and S3 is continuously streamed into Snowflake, where it is made available for immediate reporting and analysis. This capability provides accurate and timely insights, allowing the company to respond quickly to emerging trends and operational needs.
The dashboard features an intuitive, user-friendly interface designed to be accessible to users across the organization, regardless of their technical proficiency. It also includes access-based control mechanisms, allowing administrators to set different levels of access for users. This ensures that sensitive data is protected and that only authorized personnel can view or manipulate critical information within the reports.
These features collectively enable Connection Holdings to manage their extensive data operations effectively, ensuring that their reporting and compliance needs are met with precision and efficiency. The Vested Exchange dashboard not only enhances operational performance but also supports strategic decision-making across the organization.
The development and deployment of the Vested Exchange dashboard followed a structured yet flexible approach, designed to meet the evolving needs of Connection Holdings. This process involved several key phases:
The project began with an in-depth consultation with Connection Holdings' stakeholders to understand their specific challenges and objectives. Detailed discussions were held to outline the scope of the project, focusing on the need for a solution that could efficiently manage, process, and report on their vast data set across various platforms.
The dashboard’s architecture was designed using a traditional tech stack, with AWS Glue for data integration, DynamoDB for real-time data storage, and Snowflake for data warehousing. As the aged data realm evolved to include real-time data, there was a critical need for an efficient and cost-effective method of aggregating this data and moving it into Snowflake. This approach allowed Connection Holdings to perform comprehensive analysis on large data sets without the limitations imposed by individual platforms.
For example, platforms like Leadspedia limit data extraction to around 100,000 records at a time, requiring significant time and sometimes support intervention for larger data sets. By migrating data into Snowflake, we enabled Connection Holdings to bypass these restrictions and view their data holistically without having to manually access each platform.
As the project evolved, so did the system’s capabilities. The transition from managing aged data to incorporating real-time data required continuous adjustments and enhancements to the dashboard. We focused on optimizing the data pipeline to ensure that it remained both efficient and cost-effective. This ongoing evolution allowed Connection Holdings to adapt to changing business needs while maintaining robust data analysis capabilities.
Rigorous testing was conducted to ensure the accuracy, reliability, and performance of the dashboard. Connection Holdings' team members were actively involved throughout the testing phase, providing valuable feedback that was used to refine the dashboard’s functionality. Their involvement ensured that the final product was aligned with their operational needs and user expectations.
Comprehensive training sessions were conducted to equip Connection Holdings' staff with the knowledge and skills needed to fully utilize the dashboard. The training covered all aspects of the system, from basic navigation to advanced reporting features. After the training, the dashboard was rolled out across the organization, with ongoing support provided to address any issues and incorporate additional enhancements as needed.
Even after the initial rollout, the system continues to evolve. As new data sources and requirements emerge, we work closely with Connection Holdings to update and refine the dashboard. This commitment to continuous improvement ensures that the Vested Exchange dashboard remains a vital tool for managing their complex data operations.
The implementation of the Vested Exchange reporting dashboard has delivered significant results and benefits for Connection Holdings:
The automated processes and streamlined data management capabilities of the dashboard have conservatively saved Connection Holdings approximately 20 hours per week in manual data processing and reporting tasks. This time-saving has allowed the team to focus on more strategic initiatives, improving overall productivity and efficiency within the organization.
The dashboard has significantly improved Connection Holdings' ability to provide a better client experience. With real-time data updates and advanced filtering options, Connection Holdings can now deliver feedback to their publishers within minutes, rather than hours or days. This rapid turnaround time has led to better customer service, strengthening relationships with publishers and clients alike.
The comprehensive view of data provided by the dashboard, along with its advanced filtering and real-time insights, has empowered Connection Holdings to make more informed decisions. The ability to quickly analyze large datasets without the limitations of individual platforms has enabled the company to respond swiftly to market changes and operational needs, optimizing both revenue and performance.
The built-in compliance and auditing tools have reduced the risk of non-compliance and ensured that regulatory standards are consistently met. This has not only minimized potential legal risks but also enhanced the company’s reputation in highly regulated industries such as Medicare and insurance.
The implementation of the pricing and performance matrix, along with the Leads and Calls Pricing Matrix, has allowed Connection Holdings to optimize their pricing strategies based on real-time data. This has resulted in more accurate pricing, better revenue management, and improved profitability across their various verticals.
The customized reporting dashboard for Vested Exchange has transformed the way Connection Holdings manages and leverages their vast datasets. By integrating multiple data sources, automating reporting processes, and providing real-time, accurate insights, the dashboard has enhanced operational efficiency, improved compliance, and supported better decision-making. As Connection Holdings continues to grow, the Vested Exchange dashboard will play a critical role in driving their success.
To learn more about how our customized reporting solutions can benefit your organization, click the button below.
