Coding With Fun
Home Docker Django Node.js Articles Python pip guide FAQ Policy
 

How to install pyspark via python pip

pyspark - Apache Spark Python API, it belongs to Classifiers:

- Programming Language :: Python :: Implementation :: CPython
- Programming Language :: Python :: Implementation :: PyPy
- Typing :: Typed

When you know about this project and you want to new install pyspark to support your project or you get trouble as ModuleNotFoundError: No module named "pyspark" or ImportError: cannot import name "pyspark" in your project, let follow this tutorial to install pyspark

Installation:

Step 1: First, ensure you installed pip in your os, to check pip has been installed on your computer

In Windows (CMD):

py -m pip --version

In Unix/macOS:

python3 -m pip --version

Ensure pip, setuptools, and wheel are up to date:

In Windows (CMD):

py -m pip install --upgrade pip setuptools wheel

In Unix/macOS:

python3 -m pip install --upgrade pip setuptools wheel

Optional - If you want to install in virtual environment:

In Windows (CMD):

- Install virtualenv - if you installed it, please ignore

py -m pip install --user virtualenv

- Create a virtual environment

py -m venv test_pyspark_env

- Active the virtual environment

test_pyspark_env\Scripts\active

In Unix/macOS:

- Install virtualenv - if you installed it, please ignore

pip3 install virtualenv

- Create a virtual environment

python3 -m venv test_pyspark_env

- Active the virtual environment

source test_pyspark_env/bin/active

Step 2: OK, now, let flow below content to start the installation pyspark

To install pyspark on Windows(CMD):

py -m pip install pyspark

To install pyspark on Unix/macOs:

pip install pyspark

Step 3: If you want to install a specific pyspark version, add ==<pyspark version> to the end command line

Example: 

pip install pyspark==2.1.1

Please see the version list below table:

VersionReleased dateCommand
pyspark 3.3.02022-06-15T08:58:01Windows:

py -m pip install pyspark==3.3.0

Unix/macOs:

pip install pyspark==3.3.0

pyspark 3.2.22022-07-15T14:53:39Windows:

py -m pip install pyspark==3.2.2

Unix/macOs:

pip install pyspark==3.2.2

pyspark 3.2.12022-01-26T05:20:58Windows:

py -m pip install pyspark==3.2.1

Unix/macOs:

pip install pyspark==3.2.1

pyspark 3.2.02021-10-18T08:17:19Windows:

py -m pip install pyspark==3.2.0

Unix/macOs:

pip install pyspark==3.2.0

pyspark 3.1.32022-02-18T23:01:24Windows:

py -m pip install pyspark==3.1.3

Unix/macOs:

pip install pyspark==3.1.3

pyspark 3.1.22021-05-27T17:18:30Windows:

py -m pip install pyspark==3.1.2

Unix/macOs:

pip install pyspark==3.1.2

pyspark 3.1.12021-03-02T09:05:59Windows:

py -m pip install pyspark==3.1.1

Unix/macOs:

pip install pyspark==3.1.1

pyspark 3.0.32021-06-23T07:36:16Windows:

py -m pip install pyspark==3.0.3

Unix/macOs:

pip install pyspark==3.0.3

pyspark 3.0.22021-02-19T18:08:00Windows:

py -m pip install pyspark==3.0.2

Unix/macOs:

pip install pyspark==3.0.2

pyspark 3.0.12020-09-07T15:29:07Windows:

py -m pip install pyspark==3.0.1

Unix/macOs:

pip install pyspark==3.0.1

pyspark 3.0.02020-06-16T10:26:35Windows:

py -m pip install pyspark==3.0.0

Unix/macOs:

pip install pyspark==3.0.0

pyspark 2.4.82021-05-15T00:32:18Windows:

py -m pip install pyspark==2.4.8

Unix/macOs:

pip install pyspark==2.4.8

pyspark 2.4.72020-09-12T08:00:08Windows:

py -m pip install pyspark==2.4.7

Unix/macOs:

pip install pyspark==2.4.7

pyspark 2.4.62020-06-06T00:05:50Windows:

py -m pip install pyspark==2.4.6

Unix/macOs:

pip install pyspark==2.4.6

pyspark 2.4.52020-02-06T19:13:16Windows:

py -m pip install pyspark==2.4.5

Unix/macOs:

pip install pyspark==2.4.5

pyspark 2.4.42019-08-31T05:11:21Windows:

py -m pip install pyspark==2.4.4

Unix/macOs:

pip install pyspark==2.4.4

pyspark 2.4.32019-05-07T06:56:31Windows:

py -m pip install pyspark==2.4.3

Unix/macOs:

pip install pyspark==2.4.3

pyspark 2.4.22019-04-24T03:45:14Windows:

py -m pip install pyspark==2.4.2

Unix/macOs:

pip install pyspark==2.4.2

pyspark 2.4.12019-04-01T07:48:11Windows:

py -m pip install pyspark==2.4.1

Unix/macOs:

pip install pyspark==2.4.1

pyspark 2.4.02018-11-05T07:46:52Windows:

py -m pip install pyspark==2.4.0

Unix/macOs:

pip install pyspark==2.4.0

pyspark 2.3.42019-09-09T09:05:23Windows:

py -m pip install pyspark==2.3.4

Unix/macOs:

pip install pyspark==2.3.4

pyspark 2.3.32019-02-15T23:05:03Windows:

py -m pip install pyspark==2.3.3

Unix/macOs:

pip install pyspark==2.3.3

pyspark 2.3.22018-09-25T04:36:36Windows:

py -m pip install pyspark==2.3.2

Unix/macOs:

pip install pyspark==2.3.2

pyspark 2.3.12018-06-08T23:16:54Windows:

py -m pip install pyspark==2.3.1

Unix/macOs:

pip install pyspark==2.3.1

pyspark 2.3.02018-02-28T20:21:25Windows:

py -m pip install pyspark==2.3.0

Unix/macOs:

pip install pyspark==2.3.0

pyspark 2.2.32019-01-13T00:50:39Windows:

py -m pip install pyspark==2.2.3

Unix/macOs:

pip install pyspark==2.2.3

pyspark 2.2.22018-07-03T18:27:40Windows:

py -m pip install pyspark==2.2.2

Unix/macOs:

pip install pyspark==2.2.2

pyspark 2.2.12018-01-06T00:28:01Windows:

py -m pip install pyspark==2.2.1

Unix/macOs:

pip install pyspark==2.2.1

pyspark 2.2.02017-07-12T08:33:10Windows:

py -m pip install pyspark==2.2.0

Unix/macOs:

pip install pyspark==2.2.0

pyspark 2.1.32018-06-29T21:34:56Windows:

py -m pip install pyspark==2.1.3

Unix/macOs:

pip install pyspark==2.1.3

pyspark 2.1.22017-10-25T08:29:46Windows:

py -m pip install pyspark==2.1.2

Unix/macOs:

pip install pyspark==2.1.2

pyspark 2.1.12017-05-06T23:16:48Windows:

py -m pip install pyspark==2.1.1

Unix/macOs:

pip install pyspark==2.1.1

Step 4: Otherwise, you can install pyspark from local archives:

Download the distribution file from pyspark-3.3.0.tar.gz or the specific pyspark version in the below list of distribution

After that, install by command:

On Windows(CMD):

py -m pip install <path_to_pyspark_downloaded_file>

On Unix/macOs:

pip install <path_to_pyspark_downloaded_file>

List distribution:

Project link:

- Homepage

 

Related

pyspacemouse

pyspacer

pyspaces

pyspacewar

pySpaceX

pySpack

pySpam

pyspam-classifier

pyspamsum

pyspanner

pyspark

pyspark-anonymizer

pyspark-bucketmap

pyspark-cloud

pyspark-config

pyspark-ds-toolbox

pyspark-gcs