In this session we will discuss the problems that Data Engineering at the Department of City Planning encountered managing datasets and introduce the open source tooling that we’ve built to manage metadata, generate documentation, enforce data quality, and automate distribution of data to platforms, with a focus specifically on NYC Open Data.
This talk is aimed primarily at those who have an interest in automating some or all of the above. We will walk through how we, at City Planning, catalog our dataset metadata; how that metadata is used to generate READMES, data dictionaries, and other metadata files; how to leverage metadata for automated QA; how to automate distribution of data to destinations like the Tyler/Socrata open data platform, databases, FTP servers, and data lakes; and finally how interested developers can make use of our framework and potentially contribute code of their own.
This presentation is part of the Open Data @ NYC Planning event series.
Click here to RSVP for virtual attendance.
Click the blue “Going” button below to RSVP for in-person attendance at the Department of City Planning’s offices (120 Broadway, New York, NY 10271).
Click "Going" to register to attend this presentation in-person.
