Finding files in directory...
Files found in directory: 11866
Loading files info...
[10000/11866] 84.2744% | 1000 items in 0.0786392 sec | 0.786392s | Total: 0.933133s
Files info loaded
Detecting language of files...
[10000/11866] 84.2744% | 1000 items in 0.0052851 sec | 0.052851s | Total: 0.062713s
ru: 11851 99.8736%
en: 1 0.00842744%
total: 11866
Detecting language of files done
Categorizing started...
Categorizing en files
entertainment: 1 files
./source_dir/20200131/09/2434049231894601361.html
Preparing files...
files readed
[entertainment] words: 100
[entertainment] words after min filter: 98
[entertainment] pairs built
[entertainment] grouped
all threads done
[1/1] 100% | 1000 items in 0.383 sec | 0.000383s  ( groups: 0.000369s/96.3446% ) | Total: 0.000383s
done
Categorizing ru files
[5000/11851] 42.1905% | 1000 items in 0.214976 sec | 1.07488s | Total: 2.54768s
[10000/11851] 84.3811% | 1000 items in 0.206217 sec | 2.06218s | Total: 2.44388s
society: 5216 files
./source_dir/20200131/13/2576472880281630197.html
other: 284 files
./source_dir/20200131/08/6617173285108845836.html
science: 1169 files
./source_dir/20200131/17/6005656255025898036.html
entertainment: 1161 files
./source_dir/20200131/13/466060458247304354.html
economy: 1951 files
./source_dir/20200131/17/8158052408580492208.html
technology: 1185 files
./source_dir/20200131/13/9178748522376370746.html
sports: 885 files
./source_dir/20200131/16/1346747955246294543.html
Preparing files...
files readed
[technology] words: 30591
[entertainment] words: 33163
[economy] words: 32481
[science] words: 30374
[other] words: 10088
[other] words after min filter: 10086
[society] words: 61366
[sports] words: 20386
[sports] words after min filter: 20385
[other] pairs built
[other] grouped
[economy] words after min filter: 15960
[entertainment] words after min filter: 13102
[science] words after min filter: 12658
[technology] words after min filter: 12989
[society] words after min filter: 31933
[sports] pairs built
[sports] grouped
[entertainment] pairs built
[technology] pairs built
[science] pairs built
[entertainment] grouped
[technology] grouped
[science] grouped
[economy] pairs built
[economy] grouped
[society] pairs built
[society] grouped
all threads done
11592 ./source_dir/20200131/15/5818180463237396334.html
11677 ./source_dir/20200131/12/2680214431855661976.html
11476 ./source_dir/20200131/15/108680175208951967.html
11513 ./source_dir/20200131/12/7714509747591665194.html
11220 ./source_dir/20200131/09/8744129923463682643.html
11187 ./source_dir/20200131/10/2023780589539265976.html
11110 ./source_dir/20200131/13/8449331443509461700.html
11013 ./source_dir/20200131/11/1357699434221109595.html
11098 ./source_dir/20200131/14/1832059906712835079.html
10875 ./source_dir/20200131/12/7138596759900047805.html

----------------------------------
2035 ./source_dir/20200131/09/652432218275797019.html
2512 ./source_dir/20200131/13/7138596759619435338.html
2417 ./source_dir/20200131/17/319236742922817398.html
2442 ./source_dir/20200131/10/8744129923181188873.html
2033 ./source_dir/20200131/09/3181417701759007261.html

----------------------------------
426 ./source_dir/20200131/10/3759273614359970872.html
988 ./source_dir/20200131/11/3759273614659384596.html
565 ./source_dir/20200131/08/3759273613176410818.html
131 ./source_dir/20200131/13/3759273613889002664.html
587 ./source_dir/20200131/13/3759273615102981861.html
534 ./source_dir/20200131/12/3759273614568788307.html
1407 ./source_dir/20200131/09/3759273614153911233.html
1257 ./source_dir/20200131/08/3759273614701797720.html

----------------------------------
4159 ./source_dir/20200131/14/178863987852053189.html
4289 ./source_dir/20200131/14/1688417002251754239.html
3882 ./source_dir/20200131/12/4265183764879482485.html
3799 ./source_dir/20200131/13/8431099001494611646.html

----------------------------------
3322 ./source_dir/20200131/14/3294693804906985042.html
3324 ./source_dir/20200131/14/5106417366634935890.html
3317 ./source_dir/20200131/14/7223864912469636690.html
3212 ./source_dir/20200131/14/2826249225240131154.html

----------------------------------
9216 ./source_dir/20200131/11/2457611933869882227.html
9729 ./source_dir/20200131/10/4773894379295596525.html
8021 ./source_dir/20200131/08/3824421120761569760.html
9162 ./source_dir/20200131/13/121480015134829986.html
5835 ./source_dir/20200131/10/2857286274061608838.html
6405 ./source_dir/20200131/15/1894541395214502357.html
7514 ./source_dir/20200131/10/7267840628350519688.html
4650 ./source_dir/20200131/10/1688417001983448361.html
6269 ./source_dir/20200131/08/6436052638723064835.html
6810 ./source_dir/20200131/10/6621826079248594803.html
5578 ./source_dir/20200131/08/2013140190297144096.html
8144 ./source_dir/20200131/10/9173763594407905139.html
5355 ./source_dir/20200131/08/2576472880054716352.html
6147 ./source_dir/20200131/11/3531949786767311097.html
4625 ./source_dir/20200131/10/397742603623679974.html
6311 ./source_dir/20200131/10/9118270334657609333.html
5811 ./source_dir/20200131/12/8770072316842217707.html

----------------------------------
9799 ./source_dir/20200131/15/7916917744435864381.html
9956 ./source_dir/20200131/14/7427872925768515722.html
10311 ./source_dir/20200131/16/4307021367891675117.html
9989 ./source_dir/20200131/12/7037402442009008203.html

----------------------------------
[11851/11851] 100% | 1000 items in 0.709022 sec | 8.40262s  ( groups: 8.34466s/99.3102% ) | Total: 8.40262s
done
Categorizing done