101010.pl is one of the many independent Mastodon servers you can use to participate in the fediverse.
101010.pl czyli najstarszy polski serwer Mastodon. Posiadamy wpisy do 2048 znaków.

Server stats:

490
active users

#spark

0 posts0 participants0 posts today

📢 New call for proposals: #Spark enables researchers from all disciplines to test or develop novel and unconventional scientific approaches, methods, theories or ideas within a short time.
🗓️ Submission deadline: 4 March 2025.
➡️snf.ch/en/CVNR0Q5f3P32Cg9f/new

Swiss National Science Foundation (SNSF)Spark: call for proposalsThe Swiss National Science Foundation (SNSF) funds excellent research at universities and other institutions.

Just caught up with the recent Delta Lake webinar,

> Revolutionizing Delta Lake workflows on AWS Lambda with Polars, DuckDB, Daft & Rust

Some interesting hints there regarding lightweight processing of big-ish data. Easy to relate to any other framework instead of Lambda, e.g. #ApacheAirflow tasks

youtu.be/BR9oFD0QMAs

youtu.be- YouTubeEnjoy the videos and music you love, upload original content, and share it all with friends, family, and the world on YouTube.

This is a customer-facing role, so if that's not your thing, keep scrolling.

TLDR: If you know Hadoop and live close enough to Belfast to commute, you should apply.

I've posted this before, but it's been a little while #fedihire. Also, adding some additional information this time. This is my team. We are already on three continents and 6 timezones, but #Belfast is a new location for the team. I know literally nothing about the office.

I know a lot of places Hadoop is the past, and sure we see a ton of #Spark (I do not understand why that is not listed in the job description but maybe because they want to emphasis that we need hadoop expertise?). You can see all the projects we support at openlogic.com/supported-techno

It depends on how you count, as I was on two teams during tradition, but I've been on this team for over 5 years now. It's a great team. I've been with the company now right at 7 years. I cannot say how we compare to Belfast employers but this is well more than double where I have stayed at any other employer (even if you count UNC-CH as a single employer rather than the different departments, I've beat them by well over a year at this point).

My manager has been on this team for almost 15 years. His manager has been with this team for almost as long as me, but with the company much longer. His manager has been here almost as long as me (I actually did orientation with him). His manager is a her and she's been here almost as long as me. So, obviously, this is a place where people want to stay!

Our team has a lot of testosterone, but when I started, our CEO was a woman. The GM for the division is a woman.

My manager is black. The manager of our sister team is black.

I think you'll find our team and company is concerned about your work product and not how you dress, what bathroom you use, or the color of your skin.

If you take a look at our careers page, you'll see this:

Work Should Be Fun
There’s always something to look forward to as a Perforce employee: scavenger hunts, community lunches, summer events, virtual games, and year-end celebrations just to name a few.

We take that shit seriously. Nauseatingly so sometimes, lol.

Actually, we take everything on the careers page seriously, but I know from experience that some places treat support like they are a shoe sole to be worn down. Not so here. It's not all rainbows and sunshine, of course. The whole point is that the customer is having an issue! Our customers treat us with respect because management demands that they do.

------

The Director of Product Development at Perforce is searching for a Enterprise Architect (#BigData Solutions) to join the team. We are looking for an individual who loves data solutions, views technology as a lifestyle, and has a passion for open source software. In this position, you’ll get hands on experience building, configuring, deploying, and troubleshooting our big data solutions, and you’ll contribute to our most strategic product offerings.

At OpenLogic we do #opensource right, and our people make it happen. We provide the technical expertise required for maintaining healthy implementations of hundreds of integrated open source software packages. If your skills meet any of the specs below, now is the time to apply to be a part of our passionate team.
Responsibilities:

Troubleshoot and conduct root cause analysis on enterprise scale big data systems operated by third-party clients. Assisting them in resolving complex issues in mission critical environments.
Install, configure, validate, and monitor a bundle of open source packages that deliver a cohesive world class big data solution.
Evaluate existing Big Data systems operated by third-party clients and identify areas for improvement.
Administer automation for provisioning and updating our big data distribution.

Requirements:

Demonstrable proficiency in #Linux command-line essentials
Strong #SQL and #NoSQL background required
Demonstrable experience designing or testing disaster recovery plans, including backup and recovery
Must have a firm understanding of the #Hadoop ecosystem, including the various open source packages that contribute to a broader solution, as well as an appreciation for the turmoil and turf wars among vendors in the space
Must understand the unique use cases and requirements for platform specific deployments, including on-premises vs cloud vs hybrid, as well as bare metal vs virtualization
Demonstrable experience in one or more cloud-based technologies (AWS or Azure preferred)
Experience with #virtualization and #containerization at scale
Experience creating architectural blueprints and best practices for Hadoop implementations
Some programming experience required
#Database administration experience very desirable
Experience working in enterprise/carrier production environments
Understanding of #DevOps and automation concepts
#Ansible playbook development very desirable
Experience with #Git-based version control
Be flexible and willing to support occasional after-hours and weekend work
Experience working with a geographically dispersed virtual team

jobs.lever.co/perforce/479dfdd

www.openlogic.comSupported Open Source Technologies | OpenLogicOpenLogic by Perforce supports hundreds of open source technologies. Search this list to see which packages we most often support for customers.

Day 4 of 12: Understanding key terms for data professionals

As more and more data is generated, we need technologies to process it efficiently. Companies also want to be able to process data in (near) real time. This is where tools such as Spark or Kafka (Big Data Technologies) come into play.

💻 Today's Small Practical Project:
Develop a small pipeline with Python that simulates, processes and saves real-time data: For example, simulate real-time data streams of temperature values. Then check whether the temperature exceeds a critical threshold value. As an extension, you can plot the temperature data in real time.

#Spark niestety w moim flow upadł. Wersja na desktopa jest dla mnie nieużywana z racji tego, że korzystam z wielu funkcji macOS (Tagi, Skróty klawiszowe: Sent in Mail w menu, pracuje na katalogach i regułach na serwerach). Mail śmiga i się integruje, w Sparku muszę albo kombinować jak coś obejść, albo się nie da (np. z podglądu wydruku często korzystam z opcji Save to PDF i Sent in Mail i tej drugiej nie udało mi się zastąpić).
Na mobilnym urządzeniu jest lepiej do mojego flow, ale wartości dodane czyli inteligentne katalogowanie nie jest mi aż tak potrzebne jak mam reguły.
Tak, że ten. Zachwyt aplikacja ok, jest super. Przydatność - to zależy - u mnie nie daje korzyści.

@voland @michael @moridin @iMagazinePL

If you’re an Apache #Spark user, you benefit from its speed and scalability for #BigData processing.

However, you might still want to leverage #RStats’s extensive ecosystem of packages and intuitive syntax. One effective way to do this is by writing user-defined functions (UDFs) with sparklyr.

UDFs enable you to execute R functions within Spark, harnessing Spark’s processing power and combining the strengths of both tools.

🔍 Learn more in a recent blog post → posit.co/blog/databricks-udfs/

Ein 50 Jahre alter Verstärker von Fender oder Marshall oder Hiwatt ist heute ein begehrter Klassiker. Und er funktioniert (wenn gewartet) heute immer noch, wie vor 50 Jahren: Gitarre, Kabel, Strom, fertig.

Was wird aus den Amps, die heute mit #Bluetooth, #DSP und App auf den Markt kommen: wird es da in 5 Jahren noch eine angepasste und aktualisierte App für geben? Oder werden die Funktionen langsam sterben, wenn die passenden Handys verschwinden?

The sparklyr package and friends have been getting some important updates in the past few months!

sparklyr is a package that allows you to interact with Spark using familiar R interfaces, such as dplyr, broom, and DBI. You can also gain access to Spark's distributed Machine Learning libraries, Structure Streaming, and ML Pipelines from R.

Read more in the blog post: blogs.rstudio.com/ai/posts/202

Posit AI BlogPosit AI Blog: News from the sparkly-verseHighlights to the most recent updates to `sparklyr` and friends