Finding files in directory...
Files found in directory: 10380
Loading files info...
[10000/10380] 96.3391% | 1000 items in 0.176714 sec | 1.76714s | Total: 1.83429s
Files info loaded
Detecting language of files...
[10000/10380] 96.3391% | 1000 items in 0.006437 sec | 0.06437s | Total: 0.0668161s
ru: 10372 99.9229%
en: 1 0.00963391%
total: 10380
Detecting language of files done
Categorizing started...
Categorizing en files
technology: 1 files
./source_dir/20191209/11/2000617736925817824.html
Preparing files...
files readed
[technology] words: 82
[technology] words after min filter: 80
[technology] pairs built
[technology] grouped
all threads done
[1/1] 100% | 1000 items in 0.296 sec | 0.000296s  ( groups: 0.000289s/97.6351% ) | Total: 0.000296s
done
Categorizing ru files
[5000/10372] 48.2067% | 1000 items in 0.23364 sec | 1.1682s | Total: 2.42332s
[10000/10372] 96.4134% | 1000 items in 0.220962 sec | 2.20962s | Total: 2.29182s
society: 4712 files
./source_dir/20191209/09/3223989932776190580.html
technology: 1006 files
./source_dir/20191209/12/2198666504288535126.html
economy: 1270 files
./source_dir/20191209/08/770303467210717449.html
sports: 1305 files
./source_dir/20191209/17/9173763592933712465.html
other: 240 files
./source_dir/20191209/10/4621778510539178792.html
science: 741 files
./source_dir/20191209/11/3759273614426592178.html
entertainment: 1098 files
./source_dir/20191209/13/5978441743738079594.html
Preparing files...
files readed
[technology] words: 28386
[entertainment] words: 33258
[science] words: 25122
[society] words: 59975
[other] words: 9256
[other] words after min filter: 9254
[economy] words: 28151
[sports] words: 24096
[economy] words after min filter: 13043
[science] words after min filter: 9814
[other] pairs built
[entertainment] words after min filter: 12974
[other] grouped
[sports] words after min filter: 10628
[technology] words after min filter: 11685
[society] words after min filter: 31164
[science] pairs built
[science] grouped
[technology] pairs built
[entertainment] pairs built
[technology] grouped
[entertainment] grouped
[economy] pairs built
[economy] grouped
[sports] pairs built
[sports] grouped
[society] pairs built
[society] grouped
all threads done
10174 ./source_dir/20191209/12/1357699435523426408.html
10357 ./source_dir/20191209/11/8449331442846642024.html
10204 ./source_dir/20191209/13/2023780590289167940.html
10124 ./source_dir/20191209/13/8192351481035608886.html
10165 ./source_dir/20191209/15/5818180464332597036.html
10327 ./source_dir/20191209/11/7714509748735341937.html
9564 ./source_dir/20191209/15/1832059907520757732.html
9642 ./source_dir/20191209/10/8744129923640411665.html
9518 ./source_dir/20191209/11/7138596759120555226.html

----------------------------------
1418 ./source_dir/20191209/13/1236842002115772078.html
2147 ./source_dir/20191209/14/8724142283744803908.html
1861 ./source_dir/20191209/16/5959782336656971496.html

----------------------------------
1171 ./source_dir/20191209/11/2857286274557166815.html
1219 ./source_dir/20191209/09/6500716909332376242.html
765 ./source_dir/20191209/09/3824421120737032355.html
97 ./source_dir/20191209/13/6525545600124570821.html
1006 ./source_dir/20191209/13/1894541393382962222.html
273 ./source_dir/20191209/11/9173763592767736544.html
441 ./source_dir/20191209/10/4265183765229831641.html
80 ./source_dir/20191209/09/8431099000877359103.html

----------------------------------
2787 ./source_dir/20191209/16/6176042714105575427.html
3120 ./source_dir/20191209/17/1688417001564620602.html
2642 ./source_dir/20191209/16/4265183764194493592.html
2661 ./source_dir/20191209/16/7113642702368194309.html

----------------------------------
2561 ./source_dir/20191209/16/1094974872318066166.html
2586 ./source_dir/20191209/16/3499003294058413942.html
2466 ./source_dir/20191209/11/1749237432560739769.html

----------------------------------
6779 ./source_dir/20191209/12/8701191091352401240.html
7130 ./source_dir/20191209/09/108680175867403817.html
6876 ./source_dir/20191209/14/6698086526784256629.html
6852 ./source_dir/20191209/11/5568685782725528724.html
7125 ./source_dir/20191209/13/2717769049711539125.html
7253 ./source_dir/20191209/13/7103430894212489030.html
7757 ./source_dir/20191209/08/4338124334398173763.html
3356 ./source_dir/20191209/08/8835591474372106686.html
5309 ./source_dir/20191209/08/3201225782294945112.html
3648 ./source_dir/20191209/08/4311596830312784333.html
6128 ./source_dir/20191209/10/5256058911048555607.html
4844 ./source_dir/20191209/08/3006457458221618859.html
5984 ./source_dir/20191209/08/6359253559543331454.html
6311 ./source_dir/20191209/11/1301103189795241343.html

----------------------------------
8883 ./source_dir/20191209/16/7427872924590524589.html
9157 ./source_dir/20191209/13/8744883914251360612.html
9331 ./source_dir/20191209/16/8639226687248105740.html
9255 ./source_dir/20191209/14/8361979551567448945.html
9319 ./source_dir/20191209/16/9048800179556035938.html
9351 ./source_dir/20191209/13/4955690416808201763.html
8837 ./source_dir/20191209/16/1851638047746422961.html
8449 ./source_dir/20191209/13/553339377581523862.html
8062 ./source_dir/20191209/17/9173763592933712465.html

----------------------------------
[10372/10372] 100% | 1000 items in 0.719561 sec | 7.46328s  ( groups: 7.40321s/99.195% ) | Total: 7.46328s
done
Categorizing done