download github repos from search result or given repos list or given user do it by python code
$10-30 USD
Lezárt
Kiadva ekkor: több mint 3 évvel ezelőtt
$10-30 USD
Teljesítéskor fizetve
to do
windows computer , python 3.7 code download github repos from search result or given repos list or given user
example
[login to view URL]
or
[login to view URL]
use simple packages to install like BeautifulSoup and selenium , teach me how to install
not limited by next functionalities , but all what can be done manually should be done
0
Prevent blocking like
agents = cycle([r"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36 Edg/[login to view URL]",
r"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0",
r"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"])
and time delays
and etc
1
download repos to mimic manual search
2
Download all search results like: commits, code, issues, keywords/labels (for example
[login to view URL]
) - download whole repos, do not download the same repo twice
Like
Repositories
Code
Commits
Issues
DiscussionsBeta
Packages
Marketplace
Topics
Wikis
Users
Or
All for given use like
[login to view URL]
3
unzip all compressed files in repo - for all main zip algorithms all what 7zip can uncompress , all files in repo (even for many nested folders )
4
Save each language to separate folder for example ["Python", "Java", "Shell"]
5
Search for all languages or only for given set for example ["Python", "Java", "Shell"]
6
Set parameter for N best match or N most stars or N recently updated or N most commented etc
7
You put here needed functionalities
The script also supports to stop and continue the downloads from a previous left over instance
For example internet was disconnected for computer restarted
8
Log file for errors and not downloaded repos by some reason with error message for each case
If hard disk is full make it easy: just check disk size for each repo and when It less than 5 Gb , pause and ask to free dsik size and them resume?
add filtering by date . meaning to download only fresh repos , lets say after July 2019 or December 2019.
Or repos in dates range
Full functionality as in advanced search
[login to view URL]
Basic search
This search Finds repositories with…
cat stars:>100 Find cat repositories with greater than 100 stars.
user:defunkt Get all repositories from the user defunkt.
tom location:"San Francisco, CA" Find all tom users in "San Francisco, CA".
join extension:coffee Find all instances of join in code with coffee extension.
NOT cat Excludes all results containing cat.
9
Find data hidden some where, for example
[login to view URL]
[login to view URL]
[login to view URL] 330 MB
Dear Client!
I read your description carefully and I'm very interested in your project. I am specialized in python and web scraping.i have done some projects like this.i can show demo when we connect.
I am sure that I can complete our project in time. Please share the details of your project. Awaiting your positive reply.
Thank you
Greetings! Since I have got a visual over your project's description, I have some follow-up
questions to ask so to elicit the missing pieces of requirements. Please message me so we can
proceed!
I am looking forward to showcasing you that via PM.
Please provide all the technical/non-technical requirements of this project via a document or
chat.
Please check my certifications to estimate my skills.
Regards,
Saad M.
Ummm, May be my passion and learning ability makes me perfect for this project, Because whenever i've work on any type of projects i can work on that for many hours an hours. I can give you guarantee to complete the project even if i don't have that enough experience.