Automobile manufacturer
Building a Hortonworks Big Data DEV Cluster –
Operation and onboarding of use cases
Starting point
- Hadoop development clusters distributed across multiple business areas
- Increased licensing and operational costs
- Unclear and inappropriate operational processes for development
- Time-to-market for each use case is lengthy
- No multi-tenant cluster available
- Confidential data must not be processed on the existing development clusters
Procedure
- Sizing, procurement, and installation of a Hortonworks Hadoop cluster
- Setting up Kerberos on the cluster
- Planning and execution of tests for all Hadoop components
- Development of a multi-tenant HDFS and Ranger architecture
- Development of a unified service and operations model with internal billing
- Transition and stabilisation phase for 3 use cases
Funktionen/Projektergebnis
- Tested Hortonworks Hadoop DEV cluster with HDP 2.3.2
- HDFS layout architecture for the multi-tenant data lake
- Ranger access control model
- Agreed service model with standardised processes & RACI responsibilities
- Sizing sheet for use case evaluation
- Standardised "Welcome Package" for fast onboarding of each use case on the platform
- Three migrated use cases
Customer benefits
- Professional operation of the Hadoop DEV cluster
- Provision of comprehensive Big Data platform services, such as project support for onboarding, builds, operations, or changes
- Ensuring high operational quality, even for complex issues, through the integration of data engineering expertise
- Increased development efficiency through DevOps concepts
- Cost reduction through the delegation of standard tasks to an operations team (including offshore involvement)
Marco Bruno | Senior Manager / Authorised Officer