Comments on: Stop Using Production Data For Development

By: Sanchez

Sanchez — Fri, 02 Dec 2022 09:49:09 +0000

Yeah, as a data engineer I’m getting a bit sick of hearing this tbh. It’s impossible to account for every data quality situation in a data pipeline and if you can’t test on a replica of all the production data, it just means your going to end up debugging your pipelines in production, which nobody wants.

The only people arguing against using production data in development environments are IT security teams which don’t want to make life more difficult for themselves.

By: Tales From The Field Weekly Wrap Up for the Week of 10-10-2020 – SQLServerCentral

Mon, 17 Oct 2022 23:01:35 +0000

[…] a. Stop Using Production Data for Development by Thomas LaRock Twitter […]

By: Thomas LaRock

Thomas LaRock — Wed, 09 Feb 2022 18:47:33 +0000

In reply to bonisql.

If you are looking to get started on anonymizing or generating dummy data, try Python and Faker: https://medium.com/district-data-labs/a-practical-guide-to-anonymizing-datasets-with-python-faker-ecf15114c9be

HTH

By: bonisql

bonisql — Mon, 07 Feb 2022 09:07:54 +0000

Interesting reading Thomas! Could you please recommend tools for creating dummy data? 🙂

By: henry

henry — Sat, 05 Feb 2022 16:32:34 +0000

Without knowing all the intracacies of the development/production environment, project objective, type of data, etc., making a blanket statement like “Stop Using Production Data For Development” is just ridiculous.