Generate term-docId pairs with python code

I need a freelancer to generate term-docId pairs using Python code. Input Data: - The input data is in the format of sgm and txt files. Code: - I do not have a Python code to generate the term-docId pairs, but I have an existing code for a similar task that can be used as a reference. Deadline: - The preferred deadline for the project is 2 days. Skills and Experience: - Proficiency in Python programming - Experience in working with different file formats (sgm and txt) - Familiarity with generating term-docId pairs - Ability to meet tight deadlines Regarding Project:: Sub Project 1: Use reuters, the module that while there are still more documents to be processed, accepts a document as a list of tokens (omit punctuation) and outputs term-docID pairs. Instead of appending new term-docID pairings to a list, make sure you directly append the docID to the postings list for the term. You may use a hash table. No boxes required. (a) compare timing of this SPIMI inspired procedure with the naive indexer (for 10000 term-docID pairings). (b) compile an inverted index for Reuters21578 without using any compression techniques docID hint: Use the NEWID values from the Reuters corpus to make your retrieval comparable. Subproject II: Convert your indexer into a probabilistic search engine 1. using the assumptions made about independence of terms and documents etc. and 2. using the BM25 formula 3. rank the documents your index returns and 4. for a given query, return a ranked list of results. Notes: experiment with different values for the parameters k1 and b. Test queries: 1. design four test queries: (a) a single keyword query. Compare results for the same queries of Subproject I with the results for your Nave indexer (b) a multiple keyword query for Subproject I returning documents containing all the keywords (AND) for unranked retrieval (c) a multiple keywords query returning documents containing at least one keyword (OR), where documents are ordered by how many keywords they contain) (d) a query consisting of several keywords for ranking with BM25 2. run your four test queries to showcase your code and comment on the results in your report Deliverables: well documented sample runs for your queries on the information needs: (a) Democrats’ welfare and healthcare reform policies (b) Drug company bankruptcies (c) George Bush 4. any additional testing or aborted design ideas that show off particular aspects of your project.