Find Jobs
Hire Freelancers

Application to Persist Large XML Data Files into an MSSQL Database

min $50 USD / hour

Törölve
Kiadva ekkor: majdnem 6 évvel ezelőtt

min $50 USD / hour

I have a number of very large XML documents, and I'd like to have a script made in Java that will read data within the XML document, and persist it into a database with configurable batch sizes. The specific structure of the XML documents may very, but they all have the same basic parent/child structure. For example, we might have: <Parent> <Group_Child> <Child_1>Data<Child_1> <Child_2>Data<Child_2> </Group_Child> <Child_1>Data<Child_1> <Child_2>Data<Child_2> </Group_Child> </Parent> or <Parent> <Group_Child> <Child_3>Data<Child_3> <Child_4>Data<Child_4> </Group_Child> <Child_3>Data<Child_3> <Child_4>Data<Child_4> </Group_Child> </Parent> As you can see above, the first file has two columns, Child_1 and Child_2, while the second file has Child3 and Child4. The application will work as follows in each above case: 1. Parse the XML document to get the list of ALL record element names. Just as with above, we need to review within the entire document to get a cumulative, distinct list of any and all column names; 2. Create a table with the desired structure (e.g. in the above case, tables would be created with the two columns indicated; and 3. Persist all data from the XML documents into the table created. The application should accept three parameters, the first is the filename to be read, the second is batch size, and the third is target table name. For example, if my app name is PersistRMXML, and I want a batch size of 10,000, and the tablename I want to create is SomeTableName, then the command would be: java -jar PersistRMXML [login to view URL] 10000 SomeTableName Upon execution, the application will first read the input file to determine any and all possible column names (each file can contain over one million records, and so the file should not be loaded all into memory at once). STaX can be used to accomplish this. Once this is done, the new table will be created on the target database with the desired structure. The application should contain an [login to view URL] file where the target MSSQL database connection string should be configured. Once the target table is created, the application should read records in batch sizes specified above, and insert read records into memory. Once the desired number of records has been parsed from the XML document, and stored in memory, the application should then persist the data into the new table using the BulkCopy operation of MSSQL. More on Bulk Copy can be found here: [login to view URL] As a final step, the application should validate that the number of records inserted into the target table is the same record count as the number of records persisted. If not, the application should print an error to the log files indicating 'Record Persistence Incomplete - XML record count 11000 - Table record count 50' The application should use a logger, like log4j for logging, and should log all details to the log file. the application should be compiled using the Java 8 platform, and preferable will be distributed as a far jar.
Projektazonosító: 17210592

A projektről

5 ajánlat
Távolról teljesíthető projekt
Aktiválva: 6 évvel ezelőtt

Szeretne pénzt keresni?

A Freelancer oldalán történő árajánlatadás előnyei

Határozzon meg költségvetést és időkeretet
Kapja meg fizetését a munkáért
Vázolja ajánlatát
Ingyen regisztrálhat és adhat árajánlatot munkákra
5 szabadúszó adott átlagosan $52 USD/óra összegű árajánlatot erre a munkára
Felhasználó avatár
Expert software development team here. We can help you since we have huge experience in web development. Please contact
$50 USD 40 napon belül
5,0 (99 értékelés)
8,4
8,4
Felhasználó avatár
Dear Employer Hope you are doing well...!! Coming to your project scope you can hire our exp programmer for passing data into XML format. Ping me via freelancer chat box and provide me XML documents so that we can go through with our programmer and share complete understanding of project. I have read the details mentioned in project description and let me know if we can have small discussion via chat box so that we can move forward with project finalisation I will be waiting for your reply asap Warm Regards Sam
$50 USD 40 napon belül
4,9 (105 értékelés)
8,3
8,3
Felhasználó avatár
HI, I have excellent experience in PHP and Java technologies. I have executed many projects such as an e-Commerce site for a large Semi conductor company. I have recently executed project where i read through an XML and parsed it to push the data from the XML to the database. I have excellent experience in database programming.I have worked on various databases like Oracle, MYSql, Sql. I have worked on various projects like tuning, writing complex queries , or working on full fledged BI / Analytics projects I have worked in Infosys a top IT services company for more than 10 years and now i have started on my own. As part of my infosys tenure i have been working with projects related web development on various technologies. Will be able to leverage this experience to deliver this project successfully. Also recently i have customized an open source ERP for Indian GST requirements. If you can provide more details about the work then will be able to provide a more accurate pricing and timelines. Thanks Regards Vaidy
$50 USD 40 napon belül
5,0 (13 értékelés)
5,0
5,0
Felhasználó avatár
Hello! My name is Pablo, I'm a developer with 12+ years experience. I have good knowledge of J2SE and J2EE, I can have the app ready for tommorrow using STax + JAXB or Woodstox + JAXB for the data parsing, MSSQL connection and Log4j for logging. Regards
$52 USD 40 napon belül
5,0 (14 értékelés)
4,9
4,9

Az ügyfélről

UNITED STATES zászlója
SCOTTSDALE, United States
5,0
119
Fizetési mód hitelesítve
Tagság kezdete: júl. 10, 2006

Ügyfél-hitelesítés

Köszönjük! E-mailben elküldtük a linket, melyen átveheti ajándék egyenlegét.
E-mailje elküldése során valami hiba történt. Kérjük, próbálja újra.
Regisztrált Felhasználók Összes Közzétett Munka
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Előnézet betöltése
Hozzáférést adott a helymeghatározáshoz.
Belépési munkamenete lejárt, és kijelentkeztettük. Kérjük, lépjen be újra.