Suggestions
John Heaven
Global AI Ethics Leader and Sustainability Practice Lead at IEEE Standards Association
John Heaven is a Data Engineer working at Hamburger Verkehrsverbund (hvv), which is the local transport authority in Hamburg, Germany.12 His professional background includes experience with various technologies and programming languages commonly used in data engineering and analysis.1
Technical Skills
John Heaven has expertise in:
- Python
- AWS (Amazon Web Services)
- SQL
- Git
- Kubernetes
- Docker
- PySpark
- Pandas
- NumPy
- scikit-learn
These skills indicate a strong foundation in data engineering, cloud computing, and machine learning.1
Professional Background
While specific details about his career history are limited in the provided search results, we can infer that John Heaven has built a career focused on data engineering and analysis. His current role at hvv likely involves working with large datasets related to public transportation in Hamburg, potentially for purposes such as optimizing routes, analyzing passenger data, or improving overall system efficiency.
Online Presence
John Heaven maintains a professional online presence:
- His LinkedIn profile can be found under the username "johnheaven".1
- He also has a personal website (johnheaven.eu) where he shares more information about his professional background and interests.2
It's worth noting that John Heaven is based in Hamburg, Germany, which aligns with his employment at the city's local transport authority.12
Highlights
I've been experimenting with #ApacheAirflow -> a small visualisation of weather data at my place.
- A Python script pulls in weather data for my location and puts it into Google BigQuery
- Airflow triggers the Python script every ten minutes, which is in a Docker container all of its own
- A little pre-processing in BigQuery, which selects the latest row and saves it as a view
- Google Looker Studio (formerly Data Studio) pulls in the full table and the latest data for visualisation
The only caveat: AirFlow currently runs on my laptop, so there's only data as long as I leave it running! As this is just an experimental thing, I might just run it on my server with a good old-fashioned cron job in future.
Looker (weird name) is quite easy to get the hang of. Disappointingly, though, you can't present dynamic text taken from the database, which would have been good in this case as the data indeed contains text data like "rainy", "cloudy" etc.
It generally seems a bit buggy to me. In Firefox, it suddenly stopped responding to any mouse events and even in Chrome, keyboard combinations aren't blocked so Ctrl+B gives you bold text AND sets a bookmark whenever you use it. Also, I would like to have shared the document as a public link here, but that doesn't seem to work.
At least moaning about Looker makes a change from moaning about the weather...
Juuust about got my data engineering homework in before the extended deadline for #dezoomcamp.
Apparently I get an extra point for spreading the word... I can highly recommend the course so far to people looking to get a good introduction to data engineering tools and workflows!
More information here: https://lnkd.in/emdiYe9j
If you want to copy homework, you're probably too late but it's on GitHub: https://lnkd.in/emdiYe9j
