Just my blog
Blog about everything, mostly about tech stuff I made. Here is the list of stuff I'm using at my blog. Feel free to ask me about implementations.
- Mobaxterm SSH RDP FTP...
- Thunderbird Email client
- Filezilla FTP client/server
- Nirsoft Win utils
- Sysinternals Win utils
- Pi-Hole AD block by DNS
- NUT UPS manager
- Rpi MON Raspberry monitoring
- Free CAD 3D modelling
- Free Commander Far-like filemanager
- Bitwarden Password manager
- Django web framework
- celery multi-tasking
- celery-beat Celery + Django
- celery-results Celery + Django
- Pillow Python image lib
- wsgi mod Apache + Python
- requests best in WEB requests
- openpyxl make Excell docs
- p4python Perforce + Python
- paramiko SSH + Python
- pyvmomi ESXi Vcenter + Python
I'm using these libraries so you can ask me about them.
Python и vk execute = 40 тысяч постов за 3 минуты!
Да, сучка, это Python и vk execute!
Собрать 39000 постов за 3 минуты! Закончил очередную партию парсеров, теперь этот парсер умеет делать сбор постов несколько быстрее, чем раньше. В прошлый раз, одними только силами Python while - мне удалось спарсить стену с 39 тысячами постов минимум за 15-20 минут, при этом с вероятностью в 50% поймать timeout error и запускать все заново или дробить сбор постов по дате или кол-ву. Вуаля, упорство и немного логики:
Get posts from wall from VK wall.get Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 2000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 4000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 6000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 8000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 10000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 12000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 14000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 16000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 18000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 20000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 22000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 24000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 26000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 28000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 30000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 32000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 34000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 36000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 38000 Posts present: 39143| Runs: 20| Iters: 20| Offset of posts is: 40000 All posts parsed: 39143 189.69196796417236
Я хз кто меня вообще читает, вижу только кучу репостов в линкедин и некоторые в фейсбук.
Кароч, если кто читает - дайте о себе знать, если вам нужен будет мой сценарий по сбору этих постов через vk execute.
For my english speaker folks - if you're really reading my blog - just give me a shout if you need an eng. translation of my posts - will do.