multithreading - How to parallelize crontab executions to increase user base for web app based on mongodb and mysql? -
i have symfony based web application runs on mongodb , mysql backend. principal of application each user there python script runs 4-12 times day on cronjobs , populates mysql , mongodb databases. script takes between 1.5 minutes 2 minutes execute. @ moment cronjob runs on sequential basis. means script executes job , waits job end before executing next one. moment web application has new user cronjobs auto created duration of time. 24 hours in day can run limited number of cronjobs thereby, limited number of users (around 250-300)
what need if wanted host 1000 1000000 users on web application? can run script on multithread basis? means instead of waiting job finish, launch hundreds of job @ same time. way can grow user base of operations exponentially.
but, concurrency mongodb , mysql able sustain? how many jobs can execute parallelly? scheme factors need consider grow user base? need add together more machines application?
multithreading parallel-processing scalability crontab
No comments:
Post a Comment