Hello,
I want to start a career as a Data Engineer. I have read extensively about it and possess foundational skills in PostgreSQL and Pandas. Currently, I am learning Scrapy. However, I struggle to envision how these tasks translate into real job scenarios. For instance, if my chief assigns me the task of collecting prices from an online shop, and I use Scrapy to gather the data, should I create tables in Pandas or in PostgreSQL? Moreover, what if a data scientist works with MySQL or MongoDB? What is the most convenient program for the chief or clients to receive the data?
I want to start a career as a Data Engineer. I have read extensively about it and possess foundational skills in PostgreSQL and Pandas. Currently, I am learning Scrapy. However, I struggle to envision how these tasks translate into real job scenarios. For instance, if my chief assigns me the task of collecting prices from an online shop, and I use Scrapy to gather the data, should I create tables in Pandas or in PostgreSQL? Moreover, what if a data scientist works with MySQL or MongoDB? What is the most convenient program for the chief or clients to receive the data?