We will do Sentiment Analysis using the code from this repo: GitHub Note: The entire GitHub code base for this project is about 18 MB in size. And for the first time, when you run the "server.py" from Anaconda Prompt, it downloads the BERT model of size about 450 MB.Contents of YAML file for conda environment creation: env.yml
name: barissayil channels: - defaults - conda-forge - pytorch dependencies: - python==3.9 - pip - pip: - transformers==4.15.0 - pytorch - pandas - numpy - flask - flask_cors - scikit-learnRunning the command in Ubuntu Terminal
(base) ashish@ashish-Lenovo-ideapad-130-15IKB:~/Desktop$ conda env create -f env.yml Collecting package metadata (repodata.json): done Solving environment: done ==> WARNING: A newer version of conda exists. <== current version: 4.12.0 latest version: 4.14.0 Please update conda by running $ conda update -n base -c defaults conda Downloading and Extracting Packages numpy-base-1.23.1 | 5.6 MB | ### | 100% pytz-2022.1 | 194 KB | ### | 100% tzdata-2022c | 107 KB | ### | 100% numexpr-2.8.3 | 124 KB | ### | 100% ninja-1.10.2 | 8 KB | ### | 100% flask_cors-3.0.10 | 16 KB | ### | 100% certifi-2022.9.14 | 155 KB | ### | 100% libgcc-ng-11.2.0 | 5.3 MB | ### | 100% scipy-1.7.1 | 16.9 MB | ### | 100% setuptools-63.4.1 | 1.1 MB | ### | 100% libgomp-11.2.0 | 474 KB | ### | 100% numpy-1.23.1 | 11 KB | ### | 100% pip-22.1.2 | 2.5 MB | ### | 100% flask-2.1.3 | 130 KB | ### | 100% pandas-1.4.4 | 9.8 MB | ### | 100% bottleneck-1.3.5 | 115 KB | ########### | 100% scikit-learn-1.1.1 | 6.1 MB | ########### | 100% ninja-base-1.10.2 | 109 KB | ########### | 100% python-3.9.0 | 18.1 MB | ########### | 100% pyparsing-3.0.9 | 151 KB | ########### | 100% typing_extensions-4. | 42 KB | ########### | 100% typing-extensions-4. | 9 KB | ########### | 100% _openmp_mutex-5.1 | 21 KB | ########### | 100% zipp-3.8.0 | 15 KB | ########### | 100% pytorch-1.10.2 | 44.1 MB | ########### | 100% markupsafe-2.1.1 | 21 KB | ########### | 100% cffi-1.15.1 | 228 KB | ########### | 100% Preparing transaction: done Verifying transaction: done Executing transaction: done Installed package of scikit-learn can be accelerated using scikit-learn-intelex. More details are available here: https://intel.github.io/scikit-learn-intelex For example: $ conda install scikit-learn-intelex $ python -m sklearnex my_application.py Installing pip dependencies: - Ran pip subprocess with arguments: ['/home/ashish/anaconda3/envs/barissayil/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/ashish/Desktop/condaenv.6jfaxui9.requirements.txt'] Pip subprocess output: Collecting transformers==4.15.0 Downloading transformers-4.15.0-py3-none-any.whl (3.4 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 194.6 kB/s eta 0:00:00 Requirement already satisfied: numpy>=1.17 in /home/ashish/anaconda3/envs/barissayil/lib/python3.9/site-packages (from transformers==4.15.0->-r /home/ashish/Desktop/condaenv.6jfaxui9.requirements.txt (line 1)) (1.23.1) Collecting regex!=2019.12.17 Downloading regex-2022.9.13-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (769 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 770.0/770.0 kB 169.9 kB/s eta 0:00:00 Collecting requests Downloading requests-2.28.1-py3-none-any.whl (62 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 kB 178.2 kB/s eta 0:00:00 Collecting huggingface-hub<1.0,>=0.1.0 Downloading huggingface_hub-0.9.1-py3-none-any.whl (120 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 120.7/120.7 kB 128.6 kB/s eta 0:00:00 Collecting pyyaml>=5.1 Downloading PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (661 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 661.8/661.8 kB 192.7 kB/s eta 0:00:00 Requirement already satisfied: packaging>=20.0 in /home/ashish/anaconda3/envs/barissayil/lib/python3.9/site-packages (from transformers==4.15.0->-r /home/ashish/Desktop/condaenv.6jfaxui9.requirements.txt (line 1)) (21.3) Collecting filelock Downloading filelock-3.8.0-py3-none-any.whl (10 kB) Downloading tqdm-4.64.1-py2.py3-none-any.whl (78 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.5/78.5 kB 168.1 kB/s eta 0:00:00 Collecting sacremoses Downloading sacremoses-0.0.53.tar.gz (880 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 880.6/880.6 kB 144.4 kB/s eta 0:00:00 Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Collecting tokenizers<0.11,>=0.10.1 Downloading tokenizers-0.10.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (3.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB 162.9 kB/s eta 0:00:00 Requirement already satisfied: typing-extensions>=3.7.4.3 in /home/ashish/anaconda3/envs/barissayil/lib/python3.9/site-packages (from huggingface-hub<1.0,>=0.1.0->transformers==4.15.0->-r /home/ashish/Desktop/condaenv.6jfaxui9.requirements.txt (line 1)) (4.3.0) Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/ashish/anaconda3/envs/barissayil/lib/python3.9/site-packages (from packaging>=20.0->transformers==4.15.0->-r /home/ashish/Desktop/condaenv.6jfaxui9.requirements.txt (line 1)) (3.0.9) Collecting idna<4,>=2.5 Downloading idna-3.4-py3-none-any.whl (61 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 139.1 kB/s eta 0:00:00 Collecting charset-normalizer<3,>=2 Downloading charset_normalizer-2.1.1-py3-none-any.whl (39 kB) Requirement already satisfied: certifi>=2017.4.17 in /home/ashish/anaconda3/envs/barissayil/lib/python3.9/site-packages (from requests->transformers==4.15.0->-r /home/ashish/Desktop/condaenv.6jfaxui9.requirements.txt (line 1)) (2022.9.14) Collecting urllib3<1.27,>=1.21.1 Downloading urllib3-1.26.12-py2.py3-none-any.whl (140 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.4/140.4 kB 188.4 kB/s eta 0:00:00 Requirement already satisfied: six in /home/ashish/anaconda3/envs/barissayil/lib/python3.9/site-packages (from sacremoses->transformers==4.15.0->-r /home/ashish/Desktop/condaenv.6jfaxui9.requirements.txt (line 1)) (1.16.0) Requirement already satisfied: click in /home/ashish/anaconda3/envs/barissayil/lib/python3.9/site-packages (from sacremoses->transformers==4.15.0->-r /home/ashish/Desktop/condaenv.6jfaxui9.requirements.txt (line 1)) (8.0.4) Requirement already satisfied: joblib in /home/ashish/anaconda3/envs/barissayil/lib/python3.9/site-packages (from sacremoses->transformers==4.15.0->-r /home/ashish/Desktop/condaenv.6jfaxui9.requirements.txt (line 1)) (1.1.0) Building wheels for collected packages: sacremoses Building wheel for sacremoses (setup.py): started Building wheel for sacremoses (setup.py): finished with status 'done' Created wheel for sacremoses: filename=sacremoses-0.0.53-py3-none-any.whl size=895241 sha256=3aa00715128a0c0de964dd1229c0cd2704c6ddb45ef5407c4bb3e5d273808164 Stored in directory: /home/ashish/.cache/pip/wheels/12/1c/3d/46cf06718d63a32ff798a89594b61e7f345ab6b36d909ce033 Successfully built sacremoses Installing collected packages: tokenizers, urllib3, tqdm, regex, pyyaml, idna, filelock, charset-normalizer, sacremoses, requests, huggingface-hub, transformers Successfully installed charset-normalizer-2.1.1 filelock-3.8.0 huggingface-hub-0.9.1 idna-3.4 pyyaml-6.0 regex-2022.9.13 requests-2.28.1 sacremoses-0.0.53 tokenizers-0.10.3 tqdm-4.64.1 transformers-4.15.0 urllib3-1.26.12 # # To activate this environment, use # # $ conda activate barissayil # # To deactivate an active environment, use # # $ conda deactivate
Thursday, September 22, 2022
Sentiment Analysis using BERT, DistilBERT and ALBERT (Installation)
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment