Finding files in directory... Files found in directory: 7904 Loading files info... Files info loaded Detecting language of files... ru: 0 0% en: 7904 100% total: 7904 Detecting language of files done Categorizing started... Categorizing ru files Preparing files... files readed all threads done done Categorizing en files [5000/7904] 63.2591% | 1000 items in 0.317136 sec | 1.58568s | Total: 2.50665s sports: 675 files ./source_dir/20200131/09/7580308504440647197.html science: 809 files ./source_dir/20200131/08/532798824877071680.html entertainment: 2024 files ./source_dir/20200131/11/8669301694017911858.html economy: 1207 files ./source_dir/20200131/11/3364799147948808522.html technology: 884 files ./source_dir/20200131/16/7357138825690870489.html society: 2305 files ./source_dir/20200131/09/3867135408403440305.html Preparing files... files readed [sports] words: 11086 [science] words: 14312 [science] words after min filter: 14310 [technology] words: 14267 [entertainment] words: 26323 [sports] words after min filter: 11084 [society] words: 25782 [economy] words: 15832 [economy] words after min filter: 15831 [technology] words after min filter: 7299 [entertainment] words after min filter: 13160 [society] words after min filter: 13478 [sports] pairs built [sports] grouped [science] pairs built [technology] pairs built [science] grouped [technology] grouped [economy] pairs built [economy] grouped [entertainment] pairs built [entertainment] grouped [society] pairs built [society] grouped all threads done 6451 ./source_dir/20200131/15/2127367043838078551.html 6933 ./source_dir/20200131/15/7150386083449190509.html 6848 ./source_dir/20200131/14/8669301693276396678.html 6897 ./source_dir/20200131/14/2885715105503968643.html 6443 ./source_dir/20200131/14/6060938664392012316.html 6434 ./source_dir/20200131/14/6679535024770123542.html ---------------------------------- 4581 ./source_dir/20200131/12/7653256037579483983.html 4845 ./source_dir/20200131/12/4286117812996303283.html 5932 ./source_dir/20200131/13/1502508925457900941.html 4884 ./source_dir/20200131/12/6679535025873159392.html 4612 ./source_dir/20200131/12/4718288654612903417.html 5614 ./source_dir/20200131/13/4977622829090237559.html 5983 ./source_dir/20200131/17/2027555796340675764.html 5485 ./source_dir/20200131/12/4286117813122604038.html 4561 ./source_dir/20200131/13/7150386082958110188.html ---------------------------------- 7245 ./source_dir/20200131/11/8325046883322052339.html 7261 ./source_dir/20200131/11/5575934301081583347.html 7173 ./source_dir/20200131/11/6060062400506380019.html 7093 ./source_dir/20200131/11/5090408756737621747.html ---------------------------------- 734 ./source_dir/20200131/13/6913978435604761837.html 774 ./source_dir/20200131/10/7421817124878837727.html 808 ./source_dir/20200131/09/4500271767214066045.html 183 ./source_dir/20200131/11/3802011524315708694.html 111 ./source_dir/20200131/09/2027555796647374495.html ---------------------------------- 4024 ./source_dir/20200131/10/7092425147398261597.html 4026 ./source_dir/20200131/10/2090029850960048604.html 3947 ./source_dir/20200131/10/552235479732569696.html 4003 ./source_dir/20200131/14/6446904418571912651.html 3684 ./source_dir/20200131/09/8204772968676237036.html 3715 ./source_dir/20200131/09/6694993428302013613.html 3690 ./source_dir/20200131/09/7727211173989248641.html 3586 ./source_dir/20200131/10/993065930120043.html 3329 ./source_dir/20200131/09/8196011179682377576.html 3564 ./source_dir/20200131/09/5149776702475837315.html 3629 ./source_dir/20200131/09/7635722257658230458.html 3437 ./source_dir/20200131/13/7513571674847163923.html 3474 ./source_dir/20200131/10/4740742017436938777.html 3915 ./source_dir/20200131/16/8119004128540403855.html 3723 ./source_dir/20200131/10/7318238122596514902.html 3919 ./source_dir/20200131/16/3480199991270023661.html 4039 ./source_dir/20200131/16/7686550515588604445.html 3933 ./source_dir/20200131/16/675785259837791421.html 3921 ./source_dir/20200131/12/7324224458970211251.html 3430 ./source_dir/20200131/09/4778851489778355360.html 3906 ./source_dir/20200131/12/8196011178303148171.html 3831 ./source_dir/20200131/12/3480199991424033715.html 3520 ./source_dir/20200131/09/7324224460350702262.html 3382 ./source_dir/20200131/09/3480199992804524726.html 3563 ./source_dir/20200131/13/970161748584087222.html 4033 ./source_dir/20200131/17/5644198863263680360.html 3585 ./source_dir/20200131/16/6694993428021080664.html 3805 ./source_dir/20200131/16/970161749018459421.html 3857 ./source_dir/20200131/14/6679535025242605451.html ---------------------------------- 1368 ./source_dir/20200131/10/6141642774965597142.html 1945 ./source_dir/20200131/12/4125100339013590777.html 2190 ./source_dir/20200131/14/9103146373773705337.html 2556 ./source_dir/20200131/11/7421817124458259525.html 2272 ./source_dir/20200131/11/7750663361902806111.html 2286 ./source_dir/20200131/10/6412178963524372303.html 3183 ./source_dir/20200131/11/410802300759623623.html 2631 ./source_dir/20200131/12/7580308505026595838.html ---------------------------------- [7904/7904] 100% | 1000 items in 0.374578 sec | 2.96067s ( groups: 2.94228s/99.3791% ) | Total: 2.96067s done Categorizing done