'/%3e%3cpath%20d='M50.3943%2022.237V39.5876H43.5185V22.7481C43.5185%2019.2029%2042.0411%2017.3949%2039.036%2017.3949C35.7325%2017.3949%2034.0778%2019.5338%2034.0778%2023.7585V32.9759H27.2434V23.7585C27.2434%2019.5338%2025.5857%2017.3949%2022.2822%2017.3949C19.2949%2017.3949%2017.8027%2019.2029%2017.8027%2022.7481V39.5876H10.9298V22.237C10.9298%2018.6918%2011.835%2015.8754%2013.6453%2013.7877C15.5128%2011.7049%2017.9623%2010.6355%2021.0028%2010.6355C24.522%2010.6355%2027.1813%2011.9885%2028.9542%2014.6917L30.665%2017.5633L32.3788%2014.6917C34.1517%2011.9885%2036.811%2010.6355%2040.3243%2010.6355C43.3619%2010.6355%2045.8114%2011.7049%2047.6847%2013.7877C49.4931%2015.8734%2050.3963%2018.6899%2050.3943%2022.237Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_89_8'%20x1='30.5'%20y1='0'%20x2='30.5'%20y2='65'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%236364FF'/%3e%3cstop%20offset='1'%20stop-color='%23563ACC'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e)
Jons Mostovojs<p>How messy is <a href="https://social.doma.dev/tags/terraform" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>terraform</span></a> with <a href="https://social.doma.dev/tags/gcp" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>gcp</span></a>?</p><p>I'm trying to make a system where <a href="https://social.doma.dev/tags/rust" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>rust</span></a> worker ingests data of HTTP requests via <a href="https://social.doma.dev/tags/cloudrun" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>cloudrun</span></a>, and passes them into <a href="https://social.doma.dev/tags/bigtable" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>bigtable</span></a>, which uses further ingestion recipe to export the data into <a href="https://social.doma.dev/tags/bigquery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>bigquery</span></a> </p><p>I have tried to make a complete terraform declaration for this, but got into permission issues, then I have tried to make a system that generates all the artefacts like service accounts and docker image and then refers to those from terraform builds, but I almost don't see a value of doing it like that.</p><p>Does anyone have an example of <a href="https://social.doma.dev/tags/cloudrun" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>cloudrun</span></a> <a href="https://social.doma.dev/tags/cdc" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>cdc</span></a>? I am new to this and I feel really slow. </p><p><a href="https://social.doma.dev/tags/lazyweb" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>lazyweb</span></a> <a href="https://social.doma.dev/tags/askfedi" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>askfedi</span></a></p>
101010.pl is one of the many independent Mastodon servers you can use to participate in the fediverse.
101010.pl czyli najstarszy polski serwer Mastodon. Posiadamy wpisy do 2048 znaków.
Administered by:
Server stats:
501active users
101010.pl: About · Status · Profiles directory · Privacy policy
Mastodon: About · Get the app · Keyboard shortcuts · View source code · v4.4.0-alpha.5
#bigquery
0 posts · 0 participants · 0 posts today
N-gated Hacker News<p>🍕🎨 "Map of Python" is the digital dabbling of a cartographer, lost in the vast jungle of 500,000+ <a href="https://mastodon.social/tags/Python" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Python</span></a> packages, desperately trying to create art from JSON blobs. 🧩🤷♂️ But hey, at least there's <a href="https://mastodon.social/tags/BigQuery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigQuery</span></a> to save the day from the horror of actually downloading data! 🚀🧐<br><a href="https://fi-le.net/pypi/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">fi-le.net/pypi/</span><span class="invisible"></span></a> <a href="https://mastodon.social/tags/DataVisualization" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DataVisualization</span></a> <a href="https://mastodon.social/tags/Cartography" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Cartography</span></a> <a href="https://mastodon.social/tags/ArtInTech" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ArtInTech</span></a> <a href="https://mastodon.social/tags/HackerNews" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>HackerNews</span></a> <a href="https://mastodon.social/tags/ngated" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ngated</span></a></p>
Data Recce<p>❌ No more manually cross-referencing dbt docs from dev and prod</p><p>❌ No more manually checking schemas in your data warehouse</p><p>❌ No more manually comparing row-counts on models.</p><p>See a trend? </p><p>Read how an experienced data professional validates zero regression on a <a href="https://mastodon.social/tags/dbt" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>dbt</span></a> PR:</p><p><a href="https://www.linkedin.com/posts/abdelm_recce-dbt-activity-7300436694808358914-aKiF?rcm=ACoAAAVLBgkB6r61wCsOcKgFDrtf_EEEe3UjdXs" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">linkedin.com/posts/abdelm_recc</span><span class="invisible">e-dbt-activity-7300436694808358914-aKiF?rcm=ACoAAAVLBgkB6r61wCsOcKgFDrtf_EEEe3UjdXs</span></a></p><p><a href="https://mastodon.social/tags/DataEngineering" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DataEngineering</span></a> <a href="https://mastodon.social/tags/dbt" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>dbt</span></a> <a href="https://mastodon.social/tags/Data" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Data</span></a> <a href="https://mastodon.social/tags/Analytics" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Analytics</span></a> <a href="https://mastodon.social/tags/AnalyticsEngineering" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AnalyticsEngineering</span></a> <a href="https://mastodon.social/tags/SQL" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SQL</span></a> <a href="https://mastodon.social/tags/BigQuery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigQuery</span></a> <a href="https://mastodon.social/tags/OpenSource" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenSource</span></a></p>
:rss: Qiita - 人気の記事<p>【BigQuery】画像から類似画像を検索！マルチモーダルエンベディングの簡単解説<br><a href="https://qiita.com/te_yama/items/d320f81ddffae447b495?utm_campaign=popular_items&utm_medium=feed&utm_source=popular_items" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">qiita.com/te_yama/items/d320f8</span><span class="invisible">1ddffae447b495?utm_campaign=popular_items&utm_medium=feed&utm_source=popular_items</span></a></p><p><a href="https://rss-mstdn.studiofreesia.com/tags/qiita" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>qiita</span></a> <a href="https://rss-mstdn.studiofreesia.com/tags/Python" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Python</span></a> <a href="https://rss-mstdn.studiofreesia.com/tags/BigQuery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigQuery</span></a> <a href="https://rss-mstdn.studiofreesia.com/tags/GoogleCloud" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GoogleCloud</span></a> <a href="https://rss-mstdn.studiofreesia.com/tags/%E3%83%99%E3%82%AF%E3%83%88%E3%83%AB%E6%A4%9C%E7%B4%A2" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ベクトル検索</span></a></p>
Nicolas Fränkel 🇺🇦🇬🇪<p>What's the best way to wake up than to try to fix a failing GitHub Workflow, and to finally realize <a href="https://mastodon.top/tags/Google" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Google</span></a> deleted the underlying <a href="https://mastodon.top/tags/BigQuery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigQuery</span></a> table because the default expiration date is 60 days?</p><p>Can't there be a big warning when you create the table? 🤬</p><p><a href="https://mastodon.top/tags/DXFailure" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DXFailure</span></a> IMHO</p>
Guia de TI<p>Webinar de Google Cloud Fundamentals Gratuito da Fast Lane</p><p><a href="https://guiadeti.com.br/webinar-google-cloud-fundamentals-gratuito/?feed_id=10029&_unique_id=675709187aa29" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">guiadeti.com.br/webinar-google</span><span class="invisible">-cloud-fundamentals-gratuito/?feed_id=10029&_unique_id=675709187aa29</span></a></p><p><a href="https://flipboard.social/tags/bigquery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>bigquery</span></a> <a href="https://flipboard.social/tags/cloud" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>cloud</span></a> <a href="https://flipboard.social/tags/cloudcomputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>cloudcomputing</span></a> <a href="https://flipboard.social/tags/googlecloud" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>googlecloud</span></a> <a href="https://flipboard.social/tags/tecnologia" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>tecnologia</span></a><br><a href="https://guiadeti.com.br/webinar-google-cloud-fundamentals-gratuito/?feed_id=10029&_unique_id=675709187aa29" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">guiadeti.com.br/webinar-google</span><span class="invisible">-cloud-fundamentals-gratuito/?feed_id=10029&_unique_id=675709187aa29</span></a></p>
Joe<p><a href="https://functional.cafe/tags/Snowflake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snowflake</span></a> <a href="https://functional.cafe/tags/Terraform" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Terraform</span></a> and <a href="https://functional.cafe/tags/BigQuery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigQuery</span></a></p>
khushnuma<p>In the world of data science, raw data serves as the foundation for generating actionable insights. However, managing, processing, and transforming this data into a usable format requires specialized tools. </p><p><a href="https://mastodon.social/tags/ApacheSpark" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ApacheSpark</span></a> <a href="https://mastodon.social/tags/ApacheHadoop" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ApacheHadoop</span></a> <a href="https://mastodon.social/tags/Kafka" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Kafka</span></a> <a href="https://mastodon.social/tags/Airflow" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Airflow</span></a> <a href="https://mastodon.social/tags/DBT" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DBT</span></a> <a href="https://mastodon.social/tags/Presto" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Presto</span></a> <a href="https://mastodon.social/tags/SQL" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SQL</span></a> <a href="https://mastodon.social/tags/Python" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Python</span></a> <a href="https://mastodon.social/tags/PySpark" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>PySpark</span></a> <a href="https://mastodon.social/tags/Snowflake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snowflake</span></a> <a href="https://mastodon.social/tags/Redshift" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Redshift</span></a> <a href="https://mastodon.social/tags/BigQuery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigQuery</span></a> <a href="https://mastodon.social/tags/Kubernetes" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Kubernetes</span></a> <a href="https://mastodon.social/tags/Docker" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Docker</span></a> <a href="https://mastodon.social/tags/Terraform" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Terraform</span></a> <a href="https://mastodon.social/tags/Databricks" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Databricks</span></a> </p><p>read more: <a href="https://reshukhushi.wordpress.com/2024/12/03/top-data-engineering-tools-for-data-science-professionals/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">reshukhushi.wordpress.com/2024</span><span class="invisible">/12/03/top-data-engineering-tools-for-data-science-professionals/</span></a></p>
Neil Craig<p>I deployed (then had to revert) an update to the log processing pipeline which ingests our CDN access logs in Google Cloud Run then writes to BigQuery.<br>The change migrated from the "legacy" BigQuery API to the Storage API. Thought it was worth sharing some write performance improvements seen in the Storage API:<br>* P99 0.5s (30%) lower<br>* p95 155ms (25%) lower<br>* p75 58ms (20%) lower<br>* p50 43ms (43%) *higher*<br>That's on 1/2 size Run containers so higher p50 is worth it (⬇️)<br><a href="https://mastodon.social/tags/BigQuery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigQuery</span></a> <a href="https://mastodon.social/tags/GoogleCloud" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GoogleCloud</span></a> <a href="https://mastodon.social/tags/BBC" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BBC</span></a></p>
Eph Zero :emacs: 💪🏻 🚲 🤘🏻<p>Welp, I was laid off today. Anyone need an SQL monkey? </p><p>[UPDATED to include location and link to résumé] </p><p>Physical location: <a href="https://masto.snoozetown.org/tags/madison" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>madison</span></a> <a href="https://masto.snoozetown.org/tags/wisconsin" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>wisconsin</span></a> <a href="https://masto.snoozetown.org/tags/usa" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>usa</span></a> </p><p>Link to résumé: <a href="https://drive.google.com/file/d/1RkZ_buZxopuJOAgOh0Mp0ck0onF0Scex/view?usp=sharing" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">drive.google.com/file/d/1RkZ_b</span><span class="invisible">uZxopuJOAgOh0Mp0ck0onF0Scex/view?usp=sharing</span></a></p><p><a href="https://masto.snoozetown.org/tags/SQL" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SQL</span></a> <a href="https://masto.snoozetown.org/tags/bigquery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>bigquery</span></a> <a href="https://masto.snoozetown.org/tags/lookerstudio" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>lookerstudio</span></a> <a href="https://masto.snoozetown.org/tags/gcp" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>gcp</span></a> <a href="https://masto.snoozetown.org/tags/downtowork" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>downtowork</span></a> <a href="https://masto.snoozetown.org/tags/laidoff" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>laidoff</span></a> <a href="https://masto.snoozetown.org/tags/opentowork" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>opentowork</span></a> <a href="https://masto.snoozetown.org/tags/remote" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>remote</span></a> <a href="https://masto.snoozetown.org/tags/getfedihired" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>getfedihired</span></a></p>
Neil Craig<p>Long shot...<br>Is anyone using the Node JS SDK for the BigQuery Storage API with `appendRows()`?</p><p>It seems to fail *all* writes if >= 1 row passed to `appendRows()`fails which makes it unusable for me.</p><p>The "legacy" API has a `skipInvalidRows`option but I can't find one with the Storage API. I've raised a ticket but hoping someone'll know.</p><p><a href="https://mastodon.social/tags/GoogleCloud" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GoogleCloud</span></a> <a href="https://mastodon.social/tags/BigQuery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigQuery</span></a></p>
Tomasz Nurkiewicz<p>The `QUALIFY` keyword (supported in <a href="https://fosstodon.org/tags/BigQuery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigQuery</span></a>, <a href="https://fosstodon.org/tags/Snowflake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snowflake</span></a> and <a href="https://fosstodon.org/tags/H2" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>H2</span></a>) allows you to simplify <a href="https://fosstodon.org/tags/SQL" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SQL</span></a> queries with window functions by avoiding CTE/subqueries. Equivalent:</p><p>WITH salaries_ranks AS (<br> SELECT name, dept, salary,<br> ROW_NUMBER() OVER (PARTITION BY dept ORDER BY salary DESC) AS salary_rank<br> FROM employee<br>)<br>SELECT *<br>FROM salaries_ranks<br>WHERE salary_rank = 1;</p><p>vs:</p><p>SELECT *<br>FROM employee<br>QUALIFY ROW_NUMBER() OVER (PARTITION BY dept ORDER BY salary DESC) = 1</p><p>See: <a href="https://modern-sql.com/caniuse/qualify" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">modern-sql.com/caniuse/qualify</span><span class="invisible"></span></a></p>
Thom O'Connor<p>detailed and genuinely useful blog on how we built our internal data warehouse on <a href="https://mas.to/tags/ClickHouse" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ClickHouse</span></a> at ClickHouse</p><p><a href="https://clickhouse.com/blog/building-a-data-warehouse-with-clickhouse" rel="nofollow noopener" target="_blank"><span class="invisible">https://</span><span class="ellipsis">clickhouse.com/blog/building-a</span><span class="invisible">-data-warehouse-with-clickhouse</span></a></p><p><a href="https://mas.to/tags/ClickHouse" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ClickHouse</span></a> <a href="https://mas.to/tags/DataWarehouse" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DataWarehouse</span></a><br>starring<br><a href="https://mas.to/tags/Superset" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Superset</span></a> <a href="https://mas.to/tags/AppFlow" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AppFlow</span></a> <a href="https://mas.to/tags/Airflow" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Airflow</span></a> <a href="https://mas.to/tags/Redis" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Redis</span></a> <a href="https://mas.to/tags/Docker" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Docker</span></a> <a href="https://mas.to/tags/opensource" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>opensource</span></a> <a href="https://mas.to/tags/RDS" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>RDS</span></a> <a href="https://mas.to/tags/PostgreSQL" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>PostgreSQL</span></a> <a href="https://mas.to/tags/DBT" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DBT</span></a><br>and including data from<br><a href="https://mas.to/tags/Salesforce" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Salesforce</span></a> <a href="https://mas.to/tags/AWS" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AWS</span></a> <a href="https://mas.to/tags/S3" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>S3</span></a> <a href="https://mas.to/tags/BigQuery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigQuery</span></a> <a href="https://mas.to/tags/M3ter" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>M3ter</span></a> <a href="https://mas.to/tags/Segment" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Segment</span></a> <a href="https://mas.to/tags/Marketo" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Marketo</span></a></p>
Neil Craig<p><span class="h-card"><a href="https://status.steren.fr/@steren" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>steren</span></a></span> & I had a chat a while back about our serverless log processing pipeline which I built on Google Cloud using Cloud Storage, Eventarc (PubSub), Cloud Run & BigQuery. We talked a bit around scaling & other interesting stuff.<br>The comms fairies then weaved their magic & turned it into a blog post: <br><a href="https://cloud.google.com/blog/products/serverless/how-the-bbc-uses-cloud-run-and-bigquery-to-process-logs" rel="nofollow noopener" target="_blank"><span class="invisible">https://</span><span class="ellipsis">cloud.google.com/blog/products</span><span class="invisible">/serverless/how-the-bbc-uses-cloud-run-and-bigquery-to-process-logs</span></a></p><p><a href="https://mastodon.social/tags/BBC" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BBC</span></a> <a href="https://mastodon.social/tags/GoogleCloud" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GoogleCloud</span></a> <a href="https://mastodon.social/tags/CloudRun" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>CloudRun</span></a> <a href="https://mastodon.social/tags/BigQuery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigQuery</span></a> <a href="https://mastodon.social/tags/webDev" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>webDev</span></a> <a href="https://mastodon.social/tags/serverless" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>serverless</span></a></p>
Data Dave<p>Knowing the data warehouse cost for building a dbt project has really been bugging us recently, so a colleague made a proof of concept to see the cost after running dbt.</p><p>It records the bytes billed from BigQuery to a file after dbt build finishes, and then calculates the cost based on the current cost of BigQuery ($5 per TB).</p><p>I’ve written a blog post about it:</p><p><a href="https://medium.com/inthepipeline/how-to-track-your-dbt-projects-bigquery-data-warehouse-costs-in-dbt-a-poc-716068b1e2c3" rel="nofollow noopener" target="_blank"><span class="invisible">https://</span><span class="ellipsis">medium.com/inthepipeline/how-t</span><span class="invisible">o-track-your-dbt-projects-bigquery-data-warehouse-costs-in-dbt-a-poc-716068b1e2c3</span></a></p><p>Is this something folk are interested in?</p><p><a href="https://data-folks.masto.host/tags/DataOps" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DataOps</span></a> <a href="https://data-folks.masto.host/tags/BigQuery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigQuery</span></a> <a href="https://data-folks.masto.host/tags/DBT" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DBT</span></a> <a href="https://data-folks.masto.host/tags/DataEngineering" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DataEngineering</span></a> <a href="https://data-folks.masto.host/tags/AnalyticsEngineering" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AnalyticsEngineering</span></a></p>
gemelen<p><a href="https://mammut.moe/tags/gcp" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>gcp</span></a> <a href="https://mammut.moe/tags/DataEngineering" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DataEngineering</span></a> <br><a href="https://cloud.google.com/dataform" rel="nofollow noopener" target="_blank"><span class="invisible">https://</span><span class="">cloud.google.com/dataform</span><span class="invisible"></span></a> - it seems that Google released managed DBT service, integrated with managed <a href="https://mammut.moe/tags/Airflow" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Airflow</span></a> (Cloud Composer) and <a href="https://mammut.moe/tags/BigQuery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigQuery</span></a></p>
Karl Voit :emacs: :orgmode:<p><a href="https://graz.social/tags/BigData" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigData</span></a> is Dead<br><a href="https://motherduck.com/blog/big-data-is-dead/" rel="nofollow noopener" target="_blank"><span class="invisible">https://</span><span class="ellipsis">motherduck.com/blog/big-data-i</span><span class="invisible">s-dead/</span></a></p><p>Very interesting read from one of the former <a href="https://graz.social/tags/Google" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Google</span></a> <a href="https://graz.social/tags/BigQuery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigQuery</span></a> guys working with petabytes of data. </p><p>TL;DR: <a href="https://graz.social/tags/YAGNI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>YAGNI</span></a> </p><p><a href="https://graz.social/tags/MongoDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>MongoDB</span></a> <a href="https://graz.social/tags/NoSQL" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>NoSQL</span></a></p>
donmccurdy<p>BigQuery GIS visualization tool can now display much larger SQL query result sets, thanks to deckgl:</p><p><a href="https://github.com/GoogleCloudPlatform/bigquery-geo-viz" rel="nofollow noopener" target="_blank"><span class="invisible">https://</span><span class="ellipsis">github.com/GoogleCloudPlatform</span><span class="invisible">/bigquery-geo-viz</span></a></p><p>// @GCPcloud@twitter.com <a href="https://mastodon.social/tags/dataviz" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>dataviz</span></a> <a href="https://mastodon.social/tags/geospatial" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Geospatial</span></a> @GMapsPlatform@twitter.com <a href="https://mastodon.social/tags/maps" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>maps</span></a> <a href="https://mastodon.social/tags/bigquery" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>bigquery</span></a></p>
ExploreLive feeds
Mastodon is the best way to keep up with what's happening.
Follow anyone across the fediverse and see it all in chronological order. No algorithms, ads, or clickbait in sight.
Create accountLoginDrag & drop to upload