Kailas Venkitasubramanian

Announcing nationalparkscolors - Color palettes inspired by America's National Parks

Sun, 15 Feb 2026 00:00:00 +0000

“In every walk with nature, one receives far more than he seeks.” - John Muir

Deploying AI across the Research Life Cycle

Fri, 06 Feb 2026 00:00:00 +0000

Deploying AI across the research life cycle

Last summer, I gave a workshop to the staff on augmenting the research proposal process with AI. Thought I’ll sketch out what an AI-assisted research workflow might actually look like for both quantitative and qualitative work. One caveat before we get into it. This is a map of possibilities, not a prescription. What you can actually do depends on your data, your project, and what UNC Charlotte currently permits. Before trying any of this, check what the university’s Office of OneIT has published on approved tools and data classification. That guidance sets the real boundaries. This is more of a documentation of experiments that hopefully will be part of how our AI strategy takes shape.

Some Thoughts on AI-Augmented Community Research at the Charlotte Urban Institute

Tue, 11 Nov 2025 00:00:00 +0000

I thought of compiling a few thoughts on using AI at the institute or broadly in organizations similar to ours. I’ve been using AI tools in my research for over a year and I still can’t decide if I’m more excited or more unsettled but I feel it’s more of the former these days. But this ambivalence feels like the right starting point for a conversation about where we, as an institute built on community trust and research, actually want to go with this. When I say AI, I mean the generative AI kind.

Beautiful Chord Diagrams with Flourish

Tue, 08 Jul 2025 00:00:00 +0000

There’s a moment in data visualization where a chart stops being a chart and becomes a conversation starter. Chord diagrams do that reliably. Whenever I’ve dropped one into a presentation or a report, someone in the room leans forward. They want to trace the lines. They want to understand what connects to what. That reaction is exactly why I keep coming back to them.

What Is a Chord Diagram?

A chord diagram is a circular visualization that shows relationships between entities. Imagine a circle divided into arcs — each arc represents a category or group. Ribbons stretch between arcs, connecting them. The width of each ribbon encodes the magnitude of the relationship: a thick ribbon means a strong or frequent connection; a thin one means the opposite.

Embracing multilingualism in data science

Thu, 10 Apr 2025 00:00:00 +0000

Both of those efforts — reproducibility and pipelines — rest on a more basic question: which programming languages should a small research team actually use? In the previous posts of this series, I covered why reproducibility matters and how we are designing reproducible data pipelines at the UNC Charlotte Urban Institute. This post is about the layer underneath both.

Specifically, I want to argue that embracing multilingualism—fluency in both R and Python, rather than loyalty to one—has quietly done more for our team’s output than almost any other choice we’ve made.

Switching from ArcGIS to QGIS (and a bit of R too)

Tue, 18 Feb 2025 00:00:00 +0000

I have been using ArcGIS for longer than I care to admit. I started with it during my postgraduate years, and for a long time it was just the GIS software, the one everyone used, the one you learned if you wanted to do spatial analysis seriously. Our university has an enterprise license, so it has always been available, and old habits die hard.

But lately, I find myself opening it less and less. And when I do, there is usually a nagging feeling that I could be doing this in QGIS instead.

Bayesian Improved Surname Geocoding: How It Works and Where We Use It

Sun, 15 Sep 2024 00:00:00 +0000

If you work with administrative data long enough, you run into the same wall eventually: the dataset has everything you need except race and ethnicity. Hospital discharge records, voter files, tax records, benefits enrollment data — these are often rich with information about where people live, what services they use, and what outcomes they experience. But ask whether they capture race or ethnicity, and the answer is usually no, or inconsistently, or only in ways that aren’t usable.

Using tidycensus to Analyze ACS PUMS Data

Sun, 12 May 2024 00:00:00 +0000

If you’ve spent any time working with Census data, you know the drill: pull a pre-aggregated table, get median household income by county, move on. It works, and for a lot of questions, it’s exactly what you need. But sometimes the published tables just don’t cut it. What if you want to look at wage distributions for workers with specific educational credentials? Or model individual-level outcomes rather than tract-level averages? That’s where PUMS comes in — and once you start using it, it’s hard to go back.

CRDT - Data Privacy

Wed, 10 Apr 2024 00:00:00 +0000

On data privacy practice at the Charlotte Regional Data Trust

Designing Reproducible Data Pipelines for Community Research

Sat, 09 Mar 2024 00:00:00 +0000

In the first post of this series, I argued that reproducibility is not a technical luxury for community research institutions—it is an ethical and operational obligation. In this post, I want to move from philosophy to plumbing—because this is where reproducibility becomes real.

Specifically: what does it mean to design reproducible data pipelines in a community research environment?

At the UNC Charlotte Urban Institute, this question became concrete as we built the Quality of Life Explorer, developed deposit and extraction pipelines for the Charlotte Regional Data Trust, and began orchestrating workflows using Apache Airflow in an AWS environment.

Changing CRDT operations under a Cloud

Fri, 09 Jun 2023 00:00:00 +0000

The promise and peril of a large contract

Much of the previous challenges in managing the technical operations at the data trust stemmed from a lack of understanding of the scope and extent of effort for a given piece of work and having no barometer to measure productivity (or the lack of it). This meant that everyone knew that a given piece of work took 1 month to complete, everyone agreed that this delay was not acceptable,but no one really could pinpoint where the bottlenecks were and why they existed.

Revamping the Charlotte Regional Data Trust - The story so far

Mon, 22 May 2023 00:00:00 +0000

Plunging into the Data Trust black box, and Deep Cleaning the System

Sat, 20 May 2023 00:00:00 +0000

Diving into the world of administrative data and CRDT

Administrative data is messy is not much of an adage as much as it is a reality. When I took reins of managing the data infrastructure and analytical operations of Institute for Social Capital or ISC (now called the Charlotte Regional Data Trust) in the middle of 2021, messiness extended beyond data. The dysfunction was deep in how data was collected and organized, the way data operations and analyses were conducted, how information was collected from stakeholders, and how data was disseminated.

Releasing v1.0 of the Charlotte-Mecklenburg Quality of Life Explorer Data Pipeline

Thu, 16 Mar 2023 00:00:00 +0000

A successful wrap

It’s a great day today.

We completed the version 1.0 of the Charlotte-Mecklenburg Quality of Life Explorer automation project. It’s hard to put in words how exciting and rewarding this feels but I’ll try anyways.

We managed to automate most of the data and computational processes needed to generate the 80-odd quality of life indicators featured in the explorer and create functional data pipelines to serve the application. Through this work, we’ve accomplished a significant reduction of our project workload and fundamentally transformed the nature of our engagement in this project. The completion of this work also revitalizes our team’s vision to building a reproducible data science framework at the Urban Institute and a unified data ecosystem. Let me tell you how all this worked out.

On the travails of time management at Work

Mon, 13 Mar 2023 00:00:00 +0000

Time Study Goals

Find capacity for a pro-active research agenda
Identify and address bottlenecks in project management
Create better benchmarks for Project budgeting

Time Allocation

Actual Hours

Hover the plot for more details

Blog Section under Migration

Fri, 10 Mar 2023 00:00:00 +0000

Hey, I’m still migrating older content to the blog section. If you find missing or incomplete blog posts, please check back in a couple of weeks. Thanks for understanding.

Towards reproducible data science for community and policy research - An experiential roadmap

Mon, 06 Mar 2023 00:00:00 +0000

On developing a reproducible data science framework and practice at the Charlotte Urban Insitute

Haleakala

Sun, 05 Feb 2023 00:00:00 +0000

Returning to Maui in 2023

UI Reproducibility Project

Sat, 31 Dec 2022 00:00:00 +0000

Summary

Background

Diverse research backgrounds, skills and operational practices make our institute versatile and nimble to address research problems that crosses several domains. But they also enabled research analytical practices to remain fragmented and inefficient.

The Urban Institute data science team recognized the significance of reproducibility in analytical community research practice on two distinct contexts. 1) Operational efficiency via streamlined use and reuse of data, analytical tools and assets 2) developing a culture of transparency and trust that underpins reproducible research whose products become fully replicable and auditable.

Gallery of Portraits

Fri, 30 Dec 2022 00:00:00 +0000

QoL Data Pipeline Automation

Tue, 13 Dec 2022 00:00:00 +0000

Making pipelines to the quality of life explorer seamless

CRDT Data Infrastructure Project

Thu, 08 Dec 2022 00:00:00 +0000

Summary

Background

The Charlotte Regional Data Trust (formerly Institute for Social Capital) (CRDT) is an integrated data system housed in the Urban Institute that enables acces, use and governance of administrative microdata for research that addresses community issues and solutions.

The Data Trust embarked on a significant data infrastructure upgrade after recognizing the need to scale and transform the unit’s ability to serve a greater, more diverse pool of research problems and organizations. More significantly, the existing data management and analytics practice required significant revamp to efficiently fulfill data requests and conduct research projects.

UI Data and Analytics Guide

Wed, 03 Aug 2022 00:00:00 +0000

Summary

Objective(s) and Scope

The project aims to create a comprehensive guide to all operational processes of the the Urban Institute, serving as a primary point of reference for all research staff in managing data and analytical resources of the institute.

The manual will be created using Rmarkdown, a tool that allows for the creation of rich, interactive documents. The manual will be hosted as a website that can be easily updated and maintained by team members.

Charlotte Regional Data Trust - Technical Operations Manual

Sat, 06 Aug 2022 00:00:00 +0000

On how we developed the technical operations manual at the Charlotte Regional Data Trust

Reproducible Research Framework at the Charlotte Urban Institute: Why It Matters Now

Fri, 08 Jul 2022 00:00:00 +0000

In recent years, conversations about reproducibility have moved from academic journals into policy circles, foundations, and government agencies. What was once framed as a “replication crisis” in psychology has broadened into a wider concern about the credibility, transparency, and cumulative nature of scientific work across disciplines (Open Science Collaboration, 2015).

For those of us engaged in quantitative community research—especially in dynamic regional contexts like the Charlotte metropolitan area—reproducibility is not merely a philosophical concern. It is an operational one.

CRDT Anonymization and Privacy Project

Mon, 06 Jun 2022 00:00:00 +0000

Summary

Objective(s) and Scope

The project involves the development and implementation of protocols and best practices in privacy for data dissemination, including statistical disclosure control procedures of the integrated data system hosted by CRDT. This includes creating guidelines for data collection, storage, and dissemination, as well as implementing robust technical measures to prevent unauthorized access and disclosure of sensitive information.

The project will start with a comprehensive review of current privacy practices and identification of areas that need improvement. Based on this review, a set of protocols and best practices will be developed and incorporated into the organization’s data management processes. This will include the implementation of statistical disclosure control procedures to protect sensitive information. Training sessions will be conducted to educate employees on the new privacy protocols and best practices.

Make your code purr - A short intro to iterating using PURRR in R

Tue, 10 May 2022 00:00:00 +0000

Introducing ‘Purrr’

What’s programming without iterating? Purrr is a package that provides a set of tools for working with functions and vectors in R.

Built on top of the map functions (e.g., lapply(), sapply(), etc.) in base R Purrr provides a more consistent and modern syntax. Purrr is particularly useful for performing repeated operations on lists and data frames. It provides a collection of functions that make it easier to apply functions to data, iterate over lists and vectors, and manipulate data in a flexible way.

Gallatin in Winter

Thu, 05 May 2022 00:00:00 +0000

Frozen Gallatin River near West Yellowstone, Montana 2015

CRDT Technical Operations Manual

Mon, 04 Apr 2022 00:00:00 +0000

Summary

Objective(s) and Scope

The goal of this project is to create a comprehensive technical operations manual that documents all operational processes of the Charlotte Regional Data Trust The manual will serve as a single point of reference and provide clear, up-to-date information on all operational procedures.

Sahyadri - Yet another Rstudio Dark theme

Fri, 18 Mar 2022 00:00:00 +0000

Presenting Sahyadri - An RStudio theme for Syntax highlighting

Sometimes it isn’t enough to have a hundred dark themes for your text editor.So here’s one I created.

Sahyadri. Sahyadri is a mountain range that runs along the Western coast of peninsular India. The colors of this theme are inspired by monsoon forests, the incredibly diverse flora and fauna, dark undergrowth and misty slopes.

Making Data Pipelines for the Quality of Life Explorer seamless

Sun, 13 Mar 2022 00:00:00 +0000

Introduction

The idea of building a full-fledged data and computational infrastructure to support the Charlotte-Mecklenburg Quality of Life (QoL) Explorer didn’t start with an explicit demand for it. Rather, we were grappling with a deliverables backlog that had accumulated during the Covid-19 pandemic and needed to make up ground to keep the explorer up-to-date and meet deadlines.

My tryst with American Night Skies

Sun, 13 Mar 2022 00:00:00 +0000

.page-main img {
  box-shadow: 0px 0px 2px 2px rgba( 0, 0, 0, 0.2 );
  #/* ease | ease-in | ease-out | linear */
  transition: transform ease-in-out 0.6s;
}

.page-main img:hover {
  transform: scale(1.4);
}

It’s a story that began on a dark and warm summer night at the Cape Lookout National Seashore, North Carolina. Eleven years later, the romance of starlit dark skies is still as enduring as the Atlantic.

UI Analytics Ecosystem

Sun, 13 Mar 2022 00:00:00 +0000

Summary

Objective(s) and Scope

The project aims to modernize Charlotte Urban Institute’s research analytics ecosystem and improve data analysis productivity and processes.

Key Activities include:

Data Storage and Access: Organize and standardize data storage and access to ensure consistency and efficiency in data analysis.
Code Library: Develop a reusable code library to improve productivity and teamwork among data analysts.
Analytical Tools: Identify and organize tools and techniques to make the analytical process more efficient.

RSCH 8140 - Multivariate Analytical Methods

Fri, 04 Mar 2022 00:00:00 +0000

Course Summary

This course examines statistical procedures that have multiple independent and/or dependent variables, all correlated with one another to some degree. Emphases are placed on practical issues such as selecting the appropriate statistical analyses, using SPSS and R to screen and analyze data, interpreting output, presenting results, and applying these analyses in research areas. Students are trained to be critical consumers and novice producers of multivariate research.

Animated Plots with gganimate

Thu, 03 Mar 2022 00:00:00 +0000

Line Plots

Let’s accomplish making line plots that snake across the axes. I love these because they closely draw people’s attention to the change occurring (usually over time).

First, you’ll need the following libraries loaded in R to do this exercise

CRDT Data Documentation Project

Thu, 03 Mar 2022 00:00:00 +0000

Summary

Objective(s) and Scope

The project seeks to enhance the quality and completeness of CRDT’s data documentation, and establish a centralized and organized infrastructure for storing and managing metadata.

The project includes reviewing the existing metadata, developing standardized data documentation (metadata, codebook, data dictionary), and implementing a data infrastructure for storing and organizing metadata for all databases.

Expected Outcomes

Improved data documentation quality and completeness.
Consistent and standardized data documentation across all databases.
A centralized and organized infrastructure for storing and accessing metadata.

RSCH 8150 - Structural Equation Modeling

Thu, 03 Mar 2022 00:00:00 +0000

Course Summary

Structural Equation Modeling (SEM) is designed to apply general statistical modeling techniques to establish relationships among variables. SEM provides researchers with powerful techniques that takes into account the modeling of interactions, nonlinearities, correlated independents, measurement error, correlated error terms, multiple latent independents each measured by multiple indicators, and one or more latent dependents also each with multiple indicators. Empirical research articles that use structural equation modeling as a major analytic tool appear regularly in leading academic journals. The purpose of this course is to train doctoral students in both the conceptual and applied uses of SEM.

GEOG 3000 - Introduction to Data Analytics

Tue, 01 Mar 2022 00:00:00 +0000

Course Summary

Foundational course for conducting quantitative research. Essential analytical techniques and their conceptual base, and best practices of reporting and presenting analytics through visualizations.

Department

Geography and Earth Sciences

College

Liberal Arts and Sciences

Level

Undergraduate 3000/4000

Comparison Slider for figures and images with RICV

Thu, 17 Feb 2022 00:00:00 +0000

There is no end to all the magic you can do with R. I come across at least one nifty tool in the web everyday. RICV is one such awesome little visualization package that lets you compare plots using sliders.

Here’s a short introduction of RICV’s magic.

Get set up

Install the package from github and call the libraries you need to create visualizations. ggplot is my viz workhorse. I also call ggthemr because it lets me use the rust theme whose color palettes work nicely with this website theme :).

India Ink

Sat, 05 Feb 2022 00:00:00 +0000

IndiaInk is an R package project that gives ggplot color themes inspired by the national parks and landmarks of India

Charlotte Equity Indicators Project

Mon, 03 Jan 2022 00:00:00 +0000

Quantile Regression with STATA

Fri, 31 Dec 2021 00:00:00 +0000

Summary

Pre-requisites

Basic Analysis skills in STATA
Inferential Statistics / Regression

Audience

UNC Charlotte Faculty and Graduate Students

Duration

Two 2-hour sessions

Date and Location

–

Useful Resources

Applied Multilevel Modeling with STATA

Thu, 30 Dec 2021 00:00:00 +0000

Slides

Handout

Pre-requisites

Basic Analysis skills in STATA
Inferential Statistics / Regression

Audience

UNC Charlotte Faculty and Graduate Students

Duration

Two 2-hour sessions

Date and Location

–

Advanced Visualization with R

Wed, 29 Dec 2021 00:00:00 +0000

Summary

Pre-requisites

Basic skills in R

Audience

UNC Charlotte Faculty and Graduate Students

Duration

Two 2-hour sessions

Date and Location

–

Useful Resources

Latent Growth Curve Modeling with STATA

Wed, 29 Dec 2021 00:00:00 +0000

Summary

Pre-requisites

Basic Analysis skills in STATA
Inferential Statistics / Regression

Audience

UNC Charlotte Faculty and Graduate Students

Duration

Two 2-hour sessions

Date and Location

–

Useful Resources

Missing Data and Multiple Imputation

Fri, 24 Dec 2021 00:00:00 +0000

Details

Summary

Workshop aims to give hands-on experience in handling missing data issues and implementing multiple imputation method in STATA.

Pre-requisites

Basic Analysis skills in STATA
Inferential Statistics / Regression

Audience

UNC Charlotte Faculty and Graduate Students

Duration

Two 2-hour sessions

Date and Location

–

Slides

Useful Resources

Statistical Power Analysis

Thu, 02 Dec 2021 00:00:00 +0000

Summary

Pre-requisites

Basic Statistics

Audience

UNC Charlotte Faculty and Graduate Students

Duration

Two 2-hour sessions

Date and Location

–

Useful Resources

R Essentials Bootcamp

Wed, 01 Dec 2021 00:00:00 +0000

Summary

Pre-requisites

Basic Analysis skills in STATA
Inferential Statistics / Regression

Audience

UNC Charlotte Faculty and Graduate Students

Duration

Three 4-hour sessions over three weeks

Date and Location

–

Useful Resources

Evaluation of Stillbirth Among Pregnant People With Sickle Cell Trait

Wed, 24 Nov 2021 10:00:00 +0000

Retrospective cohort study finding both sickle cell trait and disease to be associated with an increased risk of stillbirth, suggesting that sickle cell carriers would benefit from additional risk assessment during pregnancy.

Institute for Social Capital - Data Privacy & Security - Present and Future

Mon, 06 Dec 2021 00:00:00 +0000

On data privacy practice at the Institute of Social Capital

CRDT Data Integration Project

Tue, 05 Oct 2021 00:00:00 +0000

Summary

Background

This is part of the CRDT Data Infrastructure project

Objective(s) and Scope

This project aims to develop improved data integration mechanisms for CRDT’s data assets, ensuring accurate and consistent data integration. This involves the development of an entity resolution framework and the implementation of probabilistic record linkage techniques to enhance the CRDT’s data integration processes.

The project will start with a comprehensive review of CRDT’s current data integration mechanisms and identification of areas for improvement. Based on this review, an entity resolution framework will be developed to ensure accurate identification of entities across different data sources. Probabilistic record linkage techniques will be implemented to link records from different data sources and provide a unified view of the data.

Harnessing electronic health records to study emerging environmental disasters: a proof of concept with perfluoroalkyl substances (PFAS)

Wed, 11 Aug 2021 10:00:00 -0500

Proof-of-concept study detailing how Electronic Health Record (EHR) data can be leveraged to study the impacts of environmental disasters like widespread exposure perfluoroalkyl substances (PFAS).

A Bayesian Hierarchical Modeling Framework for Geospatial Analysis of Adverse Pregnancy Outcomes

Tue, 11 May 2021 00:00:00 +0000

Study describing a Bayesian hierarchichal modeling framework used to explore which neighborhood-level factors and patient-level features were most informative for preterm birth and stillbirth pregnancy outcomes.

What's work like?

Mon, 12 Apr 2021 00:00:00 +0000

Dust Settles

Get in touch

Mon, 25 Feb 2019 13:38:41 -0600

** Contact page don’t contain a body, just the front matter above. See form.html in the layouts folder.

Formspree requires a (free) account and new form to be set up. The link is made on the final published url in the field: Restrict to Domain. It is possible to register up to 2 emails free and you can select which one you want the forms to go to within Formspree in the Settings tab. **

Contributors

Mon, 01 Jan 0001 00:00:00 +0000

Thank you to all the folks who have contributed both technical and creative skills to this project:

Desirée De Leon :giraffe: (designed 5 of the custom color themes, made illustrations for the workshop, and provided general aesthetic feedback along the way)
Garrick Aden-Buie :mage: (debugged headroom.js and lent his panelset.js code to the theme)
Allison Horst :dog2: (awesome illustrations of campfires, seedlings, and evergreens, as well as my R Markdown hedgehog mascot :hedgehog:)

License

Mon, 01 Jan 0001 00:00:00 +0000

My blog posts are released under a Creative Commons Attribution-ShareAlike 4.0 International License.