Practicing on standalone vm is tiring, it slows down system as well. For those who are learning or want to try spark or big data stuff they can use Google GCP Dataproc.
Google provides 300$ amount for 1 year for gcp cloud. Below are steps to create new cluster using GCP DataProc.
Google provides 300$ amount for 1 year for gcp cloud. Below are steps to create new cluster using GCP DataProc.
1. Go to https://cloud.google.com
2. Click on Try GCP Free.
3. Fill the details. Name, City, Debit/Credit card details.
They will not cut money automatically until you opt for paid version. Card addition is just for verification purpose. Click on Start My Free Trial.
4. Once Account is setup. Click on Navigation Menu and scroll down to find DataProc and click on Clusters.
5. Click on Create Cluster to go to next step.
6. Keep region other than global as global will consume free credits faster. Here I have setup 6 node cluster with below configuration.
Master 2 cores Memory 7.5 GB HardDisk 16 GB
5 Nodes 1 core Memory 3.75 GB and HardDisk 16 GB Each
Click on Create button after filling all above details.
7. you will see cluster name etc as shown in below picture.
8. Click on Cluster Name, then go to VM instances. you will see nodes over there.
9. Click on any Node the click on ssh. It will open console session where you will find Hadoop, spark etc installed.
10. After use delete the cluster as there is no option to turn off the cluster. If you keep it running free credits will be finished. So when you want to practice then create it as it takes few minutes to create cluster.
