Community Research on Kailas Venkitasubramanian

Some Thoughts on AI-Augmented Community Research at the Charlotte Urban Institute

Tue, 11 Nov 2025 00:00:00 +0000

I thought of compiling a few thoughts on using AI at the institute or broadly in organizations similar to ours. I’ve been using AI tools in my research for over a year and I still can’t decide if I’m more excited or more unsettled but I feel it’s more of the former these days. But this ambivalence feels like the right starting point for a conversation about where we, as an institute built on community trust and research, actually want to go with this. When I say AI, I mean the generative AI kind.

Embracing multilingualism in data science

Thu, 10 Apr 2025 00:00:00 +0000

Both of those efforts — reproducibility and pipelines — rest on a more basic question: which programming languages should a small research team actually use? In the previous posts of this series, I covered why reproducibility matters and how we are designing reproducible data pipelines at the UNC Charlotte Urban Institute. This post is about the layer underneath both.

Specifically, I want to argue that embracing multilingualism—fluency in both R and Python, rather than loyalty to one—has quietly done more for our team’s output than almost any other choice we’ve made.

Switching from ArcGIS to QGIS (and a bit of R too)

Tue, 18 Feb 2025 00:00:00 +0000

I have been using ArcGIS for longer than I care to admit. I started with it during my postgraduate years, and for a long time it was just the GIS software, the one everyone used, the one you learned if you wanted to do spatial analysis seriously. Our university has an enterprise license, so it has always been available, and old habits die hard.

But lately, I find myself opening it less and less. And when I do, there is usually a nagging feeling that I could be doing this in QGIS instead.

Bayesian Improved Surname Geocoding: How It Works and Where We Use It

Sun, 15 Sep 2024 00:00:00 +0000

If you work with administrative data long enough, you run into the same wall eventually: the dataset has everything you need except race and ethnicity. Hospital discharge records, voter files, tax records, benefits enrollment data — these are often rich with information about where people live, what services they use, and what outcomes they experience. But ask whether they capture race or ethnicity, and the answer is usually no, or inconsistently, or only in ways that aren’t usable.

Using tidycensus to Analyze ACS PUMS Data

Sun, 12 May 2024 00:00:00 +0000

If you’ve spent any time working with Census data, you know the drill: pull a pre-aggregated table, get median household income by county, move on. It works, and for a lot of questions, it’s exactly what you need. But sometimes the published tables just don’t cut it. What if you want to look at wage distributions for workers with specific educational credentials? Or model individual-level outcomes rather than tract-level averages? That’s where PUMS comes in — and once you start using it, it’s hard to go back.

Designing Reproducible Data Pipelines for Community Research

Sat, 09 Mar 2024 00:00:00 +0000

In the first post of this series, I argued that reproducibility is not a technical luxury for community research institutions—it is an ethical and operational obligation. In this post, I want to move from philosophy to plumbing—because this is where reproducibility becomes real.

Specifically: what does it mean to design reproducible data pipelines in a community research environment?

At the UNC Charlotte Urban Institute, this question became concrete as we built the Quality of Life Explorer, developed deposit and extraction pipelines for the Charlotte Regional Data Trust, and began orchestrating workflows using Apache Airflow in an AWS environment.

Changing CRDT operations under a Cloud

Fri, 09 Jun 2023 00:00:00 +0000

The promise and peril of a large contract

Much of the previous challenges in managing the technical operations at the data trust stemmed from a lack of understanding of the scope and extent of effort for a given piece of work and having no barometer to measure productivity (or the lack of it). This meant that everyone knew that a given piece of work took 1 month to complete, everyone agreed that this delay was not acceptable,but no one really could pinpoint where the bottlenecks were and why they existed.

Plunging into the Data Trust black box, and Deep Cleaning the System

Sat, 20 May 2023 00:00:00 +0000

Diving into the world of administrative data and CRDT

Administrative data is messy is not much of an adage as much as it is a reality. When I took reins of managing the data infrastructure and analytical operations of Institute for Social Capital or ISC (now called the Charlotte Regional Data Trust) in the middle of 2021, messiness extended beyond data. The dysfunction was deep in how data was collected and organized, the way data operations and analyses were conducted, how information was collected from stakeholders, and how data was disseminated.

Towards reproducible data science for community and policy research - An experiential roadmap

Mon, 06 Mar 2023 00:00:00 +0000

On developing a reproducible data science framework and practice at the Charlotte Urban Insitute

UI Reproducibility Project

Sat, 31 Dec 2022 00:00:00 +0000

Summary

Background

Diverse research backgrounds, skills and operational practices make our institute versatile and nimble to address research problems that crosses several domains. But they also enabled research analytical practices to remain fragmented and inefficient.

The Urban Institute data science team recognized the significance of reproducibility in analytical community research practice on two distinct contexts. 1) Operational efficiency via streamlined use and reuse of data, analytical tools and assets 2) developing a culture of transparency and trust that underpins reproducible research whose products become fully replicable and auditable.

UI Data and Analytics Guide

Wed, 03 Aug 2022 00:00:00 +0000

Summary

Objective(s) and Scope

The project aims to create a comprehensive guide to all operational processes of the the Urban Institute, serving as a primary point of reference for all research staff in managing data and analytical resources of the institute.

The manual will be created using Rmarkdown, a tool that allows for the creation of rich, interactive documents. The manual will be hosted as a website that can be easily updated and maintained by team members.

Charlotte Regional Data Trust - Technical Operations Manual

Sat, 06 Aug 2022 00:00:00 +0000

On how we developed the technical operations manual at the Charlotte Regional Data Trust

CRDT Data Documentation Project

Thu, 03 Mar 2022 00:00:00 +0000

Summary

Objective(s) and Scope

The project seeks to enhance the quality and completeness of CRDT’s data documentation, and establish a centralized and organized infrastructure for storing and managing metadata.

The project includes reviewing the existing metadata, developing standardized data documentation (metadata, codebook, data dictionary), and implementing a data infrastructure for storing and organizing metadata for all databases.

Community Research on Kailas Venkitasubramanian

Some Thoughts on AI-Augmented Community Research at the Charlotte Urban Institute

Embracing multilingualism in data science

Switching from ArcGIS to QGIS (and a bit of R too)

Bayesian Improved Surname Geocoding: How It Works and Where We Use It

Using tidycensus to Analyze ACS PUMS Data

Designing Reproducible Data Pipelines for Community Research

Changing CRDT operations under a Cloud

The promise and peril of a large contract

Plunging into the Data Trust black box, and Deep Cleaning the System

Diving into the world of administrative data and CRDT

Towards reproducible data science for community and policy research - An experiential roadmap

UI Reproducibility Project

Background

UI Data and Analytics Guide

Objective(s) and Scope

Charlotte Regional Data Trust - Technical Operations Manual

CRDT Data Documentation Project

Objective(s) and Scope

Expected Outcomes

What's work like?

Dust Settles