Tag: Data Engineering

Graphic with bright orange background and white text that reads School of Data.

NYC School of Data is a community conference that demystifies the policies and practices around open data, technology, and service design. This year’s conference helps conclude NYC Open Data Week and features 30+ sessions organized by NYC’s civic technology, data, and design community! Our conversations and workshops will feed your mind and inspire you to improve your neighborhood.

To attend, you need to purchase tickets. The venue is accessible, and the content is all-ages friendly! If you have accessibility questions or needs, please email us at schoolofdata@beta.nyc.

Thank you to Reinvent Albany and Esri for helping to cover conference costs and making it possible to meet in 2025.

And If you can’t join us in person, tune into the main stage live stream provided by the Internet Society New York Chapter. Follow the conversation #nycsodata on Bluesky.

Purchase your tickets here.

Open-source is transforming the data engineering space. By combining tools like Parquet, Polars, DuckDB, and Dagster, data product creation can achieve a collective 1000x improvement in cost, performance, and simplicity. Plus — thanks to LLMs, it has never been easier to quickly learn how to build with these tools!

Join Christian Casazza, Data Engineer, where he’ll speak about the Open Data Stack. He’ll show you how to use open-source tools to ingest and store any dataset from the Open Data API, run a SQL transformation pipeline, and visualize the results as a live web app (all for free from your computer!). Then, learn how to work with an LLM like ChatGPT to write ETL code, build SQL queries, and create frontend apps.

Power Query is perhaps the most useful but also most unknown of Microsoft programs. Ryan Yeung, Director of Performance Evaluation and Analytics, and Lori Lam, Data Analyst, from the Department of Citywide Administrative Services (DCAS) will demonstrate how to connect Power Query to the NYC Open Data and how to automate tedious Extract, Transform, Load (ETL) processes for use in Microsoft Excel or Microsoft PowerBI.