Senior C++ Software Engineer, Voltron Data
Storage
$140-165k
+ Equity
Enterprise support for Apache Arrow
Job no longer available
Enterprise support for Apache Arrow
101-200 employees
Job no longer available
$140-165k
+ Equity
101-200 employees
Company mission
To bring the best of open-source together and build bridges across the data analytics ecosystem with Apache Arrow.
Role
Who you are
- Strong experience developing in C++, especially using Modern C++
- Experience developing and using various data lake storage technologies as: S3, Google Compute Storage, Azure Blob Storage
- Building and using distributed networked file systems such as HDFS or Ceph
- Experience working with technologies such as io_uring, DMA, RDMA, or GPUDirect Storage
- Experience with different data storage file formats such as ORC, Parquet, and Avro
- Experience with data lake table formats such as Iceberg, Delta Lake, and Hudi
What the job involves
- Upon joining:
- Spending time learning about the Apache Arrow memory layout, compute primitives, and APIs
- Familiarizing yourself with the different partners for compute kernels and the query execution engine on Apache Arrow
- Learning and embracing the Apache development process
- Within a month:
- Implementing new high-performance storage and I/O primitives
- Benchmarking existing I/O library functions to determine where there are bottlenecks
- Discovering and implementing optimizations in data reads and writes
- Participating in peer code review of all PRs related to file storage and interacting with different filesystems
- Contributing to technical discussions and technical design documents
- Within 6 months:
- Developing a comprehensive set of low level benchmarks for I/O functions targeting various local, networked and cloud storage technologies to enable monitoring for performance regressions
- Ensuring that all filesystems interactions are compatible and performant across platforms (Linux, MacOS, and Windows)
- Identifying and building reusable software components to ensure a high quality and maintainable codebase
- Within 12 months:
- Analyzing I/O throughput in a massively parallel and distributed query engine to identify inefficiencies and crafting solutions to tackle those inefficiencies
- Ensuring that the everything related to storage is built as high quality as possible, balancing performance, usability, and maintainability across the Voltron Data and Apache Arrow ecosystems
Our take
Voltron Data offers enterprise support to companies using the increasingly popular open source Apache Arrow software development platform. Its flagship service is designed specifically for companies that wish to leverage the power of Apache Arrow, but are unable or unwilling to handle the intricacies of Arrow.
The strong links to Apache Arrow are easy to explain, as co-founders Wes McKinney and Joshua Patterson were heavily involved in the creation and rise in popularity of the Apache Arrow project. In a way, a major part of Voltron Data’s remit is to promote the software development platform, while making it more palatable and user-friendly for customers.
With the likes of Google and Microsoft now adopting Apache Arrow into their own projects, Voltron Data has been able to attract incredible amounts of investment, despite only launching its first commercial product in early 2022. Voltron Data has focused on growing all aspects of its team and acquired real-time AI platform startup Claypot AI in 2024, indicating that it's maintaining its momentum.
Kirsty
Company Specialist at Welcome to the Jungle
Insights
1% employee growth in 12 months
Company
Funding (2 rounds)
Feb 2022
$88m
SERIES A
Feb 2022
$22m
SEED
Total funding: $110m
Company benefits
- Work from Anywhere - Payroll and Benefits in 150+ Countries
- Unlimited PTO
- Medical, Dental, and Vision
- Retirement [USA Only]
- Home Office Budget
- Continuing Education Budget
Company HQ
Old Mountain View, Mountain View, CA
Leadership
Joshua Patterson
(CEO)Former Data Science Principal at Accenture Technology Labs, former Senior Director for Rapids.ai, an end to end platform for accelerating data science on NVIDIA GPU.
Wes McKinney
(Senior Advisor)Before co-founding Voltron Data, they were a part of the Project Management Committee for the creation of the Apache Arrow project. Creator of the Python pandas project, which made Python the go-to language for data science, machine learning and AI.
Share this job