In my experience $25 for 1TB of ETL is outrageously high. That’s 100 cores of GCP or AWS for 4-5hrs. I have some very very CPU intensive jobs that would take about that time for 300GB of data. But if it’s just parsing CSV / JSON and doing some joins then I’d think $5/TB would be more reasonable.
I agree a warehouse could get more expensive, but that’s where Athena or BigQuery is supposed to come in? BigQuery is rather expensive though. It can’t support reading Parquet natively (of course Google insists on their own format) so you have to pay their heavy tax versus just object storage and elastic compute.
Also the classic BigQuery UI is way less buggy than the new one, and the PMs won’t take bugs for the new UI.
I agree a warehouse could get more expensive, but that’s where Athena or BigQuery is supposed to come in? BigQuery is rather expensive though. It can’t support reading Parquet natively (of course Google insists on their own format) so you have to pay their heavy tax versus just object storage and elastic compute.
Also the classic BigQuery UI is way less buggy than the new one, and the PMs won’t take bugs for the new UI.