
Derk van Veen
Database specialist @ Adyen
Derk van Veen is a database specialist at Adyen. Adyen has some very large PostgreSQL databases under continuous high pressure. An important part of his job is to find creative solutions for tough problems. Whenever he, or his team, finds interesting solutions he tries to share the lessons learned with the PostgreSQL community.
POSETTE 2025 Talk
Leveraging table partitioning for query performance and data archiving
Table partitioning has become a known technique in the PostgreSQL world. There has been a talk about it at most of the conferences I visited last year. But often we talk about the technicalities of partitioning and not why we partition.
This talk will be about why table partitioning can be done. We use it for PII compliance, improving query performance by using enable_partitionwise_join and enable_partitionwise_aggregate and transparent data archival with respect to the application. Especially the data archival part is something I am really proud of.
All these techniques may sound easy, but there is a lot of ground to cover before it works to perfection. Think about finding your leading figure, denormalise tables and aligning partition boundaries. It can be a lot of work but we managed to improve query performance over 50 times for some queries!
And with all the partitioning in place we can start moving data out of our primary database into a secondary archival system without the application being aware of it. Now our mean PostgreSQL database becomes smaller, easier to manage and even faster.
Speaker
Interview
About the Speaker
-
Tell us about yourself: career, family, passions
My name is Derk van Veen, Database engineer at Adyen. I am married to Tamara and have two daughters. Besides family and work I like to play chess and diabolo.
-
What is your icebreaker for PostgreSQL events?
I am always traveling with my chess board and clock.
-
What would you say is your superpower?
My superpower is to make complicated topics very easy. I try to do the same with all the talks I give.
-
Which book are you reading right now and why did you choose it?
Resonate by Nancy Duarte. She is a great speaker, and the book has so many lessons for designing great talks.
-
What is your favorite hobby?
I love playing chess and diabolo.
About the Talk
-
Tell us about your talk. Why did you choose this topic?
Building a great partitioning framework is my biggest effort within Adyen. We are still expanding the framework, and I am learning more each week. I keep sharing my lessons and mistakes with the community so everybody can leverage the benefits of partitioning without learning all the lessons themselves.
-
Who would benefit the most from your talk and why?
I would say the biggest benefits are for those managing environments with table around and over 100GB. For me that is the tipping point for when table partitioning becomes an important topic to master.
-
What existing knowledge should an attendee have?
Prior knowledge is not required. All the topics are not complicated at all. The value is in the lessons I share; the technicalities are not difficult at all.
-
Which other talk at this year’s conference would you like to watch and why?
I am looking forward to the UUID talk from Chris Ellis. The UUID is my new favorite data type, and I am curious what he will tell about it.
-
How do you balance technical depth with engaging storytelling in your conference presentations?
When I started speaking publicly, my talks were very technically heavy with little engagement. More like mind blowing. But from Remco Claassen I learned humans can only remember 5 +- 2 things from a presentation. Since then, I have tried to have more engaging talks while focusing less on the content. I pick one, or at most 3 lessons and I spend all my efforts on imprinting these lessons. Story telling is the most effective technique to engage the audience and make the message stick. Let’s just say, I am still learning and improving when it comes to balancing technical depth with storytelling.
About PostgreSQL
-
What is your favorite PostgreSQL feature?
That must be partitioning :)
-
What is the single thing that you think differentiates PostgreSQL most from other databases?
It is open source with a great community.
-
What is your favorite PostgreSQL extension or tool? And why?
Pg_stat_statements. Nobody is running PostgreSQL without it, as far as I know.
-
What are your favorite resources for learning about PostgreSQL?
I love conferences. They give the opportunity to watch talks and meet people with the knowledge you can use.
-
If you had a magic wand, what single thing would you change in PostgreSQL as it is today?
I would add global indexes. I know they are hard to build, but they would make my life so much easier.
About POSETTE & Events
-
Have you enjoyed previous POSETTE (formerly Citus Con) conferences, either as an attendee or as a speaker? If so, what did you enjoy most about it?”
Yes, I attended both as speaker and an attendee. It is a very low barrier conference to see great talks.
-
What motivated you to speak at this year’s POSETTE: An Event for Postgres?
It has a good name, and it is online. Presenting online doesn’t give me the same feeling as compared to speaking in front of a big audience, but the impact can be huge as it is available to everyone.
-
What other PostgreSQL events in 2025 are you excited about and why?
PGDay Lowlands is going to be amazing this year! We are organizing it for the second time and this year it is taking place in Rotterdam Zoo, close to real elephants.
-
What advice would you give to fellow speakers preparing for a PostgreSQL conference?
Realize that presenting is a humbling role. You are not there for your ego, but you are there to share. The audience is the real hero, and it is your job to make their effort in listening to you worth it. Be aware of who your audience is and what they need to know before your presentation is over
-
What would be helpful to know for a first-time speaker?
Everybody is nervous before speaking. It never stops.
Past Talks
-
Partitioning your Postgres tables for 20X better performance (POSETTE 2024)
Podcast Appearances
-
From developer to PostgreSQL specialist with Derk van Veen
Join the conversation
Use the hashtag #PosetteConf