Multi-Format Candidate Data Parser and Normalizer

Mid July, 2026

Problem Statement Eightfold gets candidate data from several sources. As the data from different sources can be in different formats, we need to convert data from multiple sources to a format that is specific to Eightfold. Assume candidates data is stored as json documents in different folders (based on the source schema) in the local file system. The schema of the candidate json documents across folders is different but it is the same within a folder. Assume that within the same folder the documents are segregated based on customers and different (Eightfold) customers can follow the same schema. Design and implement a software to read these documents in different formats and transform them into a consistent format and store them in a different output folder. Example input document formats are given at the end of this document Consider the following while implementing the same Prepare a generic output format for the candidate New documents can be added in any input folders any time Existing documents can be updated in the input folders any time (schema of document within a folder does not change) Old documents can be deleted Periodically check the input folders pick up the changes and update the document in output folder A new folder with a different input format can be introduced any time, consider picking them up and transform to output format without code changes Consider the number of documents will be huge and the sync lag time between input and output should be as small as possible Consider scenarios where the input format can be XML or any other format in future Input formats Format 1: { "first_name" : "John", "last_name" : "Michel", "education" : [ { "type" : "Masters Degree", "school" : "San Jose State University", "country" : "USA", "state": "CA", "primary_subject" : "Computer science", "other_subjects" : ["Maths", "Business Accounts"] }, { "type" : "Bachelor's Degree", "school" : "San Jose State University", "country" : "USA", "state": "CA", "primary_subject" : "Maths", "other_subjects" : ["Computer Science", "Physics"] } ], "experience": [ { "from" : "10-11-2002", "title" : "Software Engineer" "to" : "12-06-2006", "company": "IBM", "country" : "USA", "state" : "CA", "city": "SFO" }, { "from" : "20-06-2006", "to" : "12-11-2021", "title" : "Senior Software Engineer" "company": "Microsoft", "country" : "USA", "state" : "CA", "city": "SFO" } ], "skills" : ["java", "python", "Databases", "Distributed Systems"] } Format 2: { "full_name": "John, Michel", "education": [ { "type": "Masters", "school": { "name": "California University of Pennsylvania", "country": "USA", "state": "CA" }, "specialization": "Computer science", "ancillary_subjects": [ "Maths", "Business Accounts" ] }, { "type": "Bachelors of Enginnering", "school": { "name": "Indian Institute of Technology", "country": "INDIA", "state": "Tamil Nadu" }, "primary_subject": "Computer Science", "other_subjects": [ "Maths", "English" ] } ], "experience": [ { "start": "15-July-2001", "end": "12-September-2005", "job_title" : "SE II", "orgnanization": { "name": "google", "country": "USA", "state": "CA", "city": "SFO" } }, { "start": "01-October-2005", "end": "11-March-2021", "job_title" : "Principal Engineer", "orgnanization": { "name": "uber", "country": "USA", "state": "CA", "city": "SFO" } } ], "skills": [ { "category" : "programming_language", "values": ["Go", "python"] }, { "category" : "databases", "values": ["MySQL", "Postgres", "Oracle"] }, { "category" : "others", "values": ["Software Engineering", "Distributed Systems", "Cloud services", "Datastructures", "Algorithms"] } ] }