Data Infrastructure on Kailas Venkitasubramanian

Revamping the Charlotte Regional Data Trust - The story so far

Mon, 22 May 2023 00:00:00 +0000

Releasing v1.0 of the Charlotte-Mecklenburg Quality of Life Explorer Data Pipeline

Thu, 16 Mar 2023 00:00:00 +0000

A successful wrap

It’s a great day today.

We completed the version 1.0 of the Charlotte-Mecklenburg Quality of Life Explorer automation project. It’s hard to put in words how exciting and rewarding this feels but I’ll try anyways.

We managed to automate most of the data and computational processes needed to generate the 80-odd quality of life indicators featured in the explorer and create functional data pipelines to serve the application. Through this work, we’ve accomplished a significant reduction of our project workload and fundamentally transformed the nature of our engagement in this project. The completion of this work also revitalizes our team’s vision to building a reproducible data science framework at the Urban Institute and a unified data ecosystem. Let me tell you how all this worked out.

CRDT Data Infrastructure Project

Thu, 08 Dec 2022 00:00:00 +0000

Summary

Background

The Charlotte Regional Data Trust (formerly Institute for Social Capital) (CRDT) is an integrated data system housed in the Urban Institute that enables acces, use and governance of administrative microdata for research that addresses community issues and solutions.

The Data Trust embarked on a significant data infrastructure upgrade after recognizing the need to scale and transform the unit’s ability to serve a greater, more diverse pool of research problems and organizations. More significantly, the existing data management and analytics practice required significant revamp to efficiently fulfill data requests and conduct research projects.