Custom Wordpress Plugin to check for Duplicate Pending Posts by Custom Field, and Publish them if they are Unique.

I have a localhost wordpress installation with an RSS-to-Post plugin that scrapes 100+ news sources and puts links to their articles into my wordpress site as posts. I have 100,000 unpublished posts, they are all "pending" status. Many of these posts are duplicates. All of the posts have a custom field called "wpe_sourcepermalink" which contains the URL link (example: "[login to view URL]") that leads back to the article's page. Any Post with the same url in the custom field "wpe_sourcepermalink" as another Post is a duplicate and needs to be deleted. Not moved to trash, but deleted. Some of the URLs in "wpe_sourcepermalink" also need to be cleaned up before being compared to each other. I need a plugin to: (0) I can set which custom field to check in the options/settings. (1) Plugin runs every 5 minutes. (2) Go through all of the "Pending" status Posts and look at the "wpe_sourcepermalink". If there is a "&" in the url, the first "&" and everything after it needs to be deleted. Example: "[login to view URL]" should become "[login to view URL]". Be careful some URLs might have multiple "&" in them. Find the first "&" and delete everything after it, so only the clean URL is left. (3) Compare the custom-field "wpe_sourcepermalink" of the Pending Posts to the "wpe_sourcepermalink" of the Published Posts. If a Published Post has that same URL, then delete the Pending Post. If no Published Post has that same URL, then make the Pending Post a Published Post. EXAMPLE: - I have 100,000 pending posts, and 0 published posts - Go to the first Pending Post. - Found Pending Post with title "Twitter erupts with flamethrower memes as Bolsonaro v DiCaprio feud over Amazon flares up" - check the custom field wpe_sourcepermalink for a "&" symbol in the url string. - IF it has a "&" symbol in the URL, go to the first "&" in the string and delete it and everything after it. - IF it does not then leave the URL alone. - Found "[login to view URL]" in the custom field wpe_sourcepermalink. - removing "?utm_source=rss&utm_medium=rss&utm_campaign=RSS" from custom field wpe_sourcepermalink. - custom field wpe_sourcepermalink is now a clean url string "[login to view URL]" - Does any Published Post have "[login to view URL]" in its custom field wpe_sourcepermalink? - IF yes, then the current pending post is a duplicate, delete it. - IF no, then the current pending post is unique, Publish it by setting it's status to "PUBLISHED". - Move on to next Pending Post. - Found Next Pending Post with title "Twitter erupts with flamethrower memes as Bolsonaro v DiCaprio feud over Amazon flares up". - check the custom field wpe_sourcepermalink for a "&" symbol in the url string - Found "[login to view URL]" in the custom field wpe_sourcepermalink. - customfield wpe_sourcepermalink does not have a "&" in it, the URL string is clean. move to next step. - Does any Published Post have "[login to view URL]" in its custom field wpe_sourcepermalink? - Yes, there is already a Published Post with "[login to view URL]" in its custom field wpe_sourcepermalink. - Delete the current Pending Post. - etc.... Unless you have any suggestions, I think this is the best way to design the plugin. I have about 1,000+ pending posts added a day that need to be checked constantly for duplicates so the plugin needs to be able to handle running every 5 minutes.