Store and Process Big Data. Excel, MySQL, Spark.

This course is intended to be an initiation to learn #BigData and #MachineLearning with #Python programming for absolute beginners that have no background in programming.

In this course, we will step by step, using the example of real data, we will go through the main processes related to the topic "Big data and machine learning".
Since the material turned out to be voluminous, I divided the course into five parts.

⇉ In this part we will consider the main options for storing big data.

⇉ In practical lesson we will install the MySQL server on computer and learn how to work and edit MySQL databases.
In the fifth lesson we will take one regular exel table and transfer the information from this table to the MySql server.

⇉ Then we will install the spark in order to work with datasets in a distributed manner.Then, to process the distributed data, we export the data from MySQL into spark. And with the help of Jupiter Notebook, we prepare the data for visualization of this data.

⚐ In this course, you will be guided through the basics of using:

• Spark

• MySQL

• Jupyter Notebook

• Spark

  
  

⚐ Topics covered in this course:

Lecture 2. Big Data Storage. Three ways to store digital data.

• What is big data?

• What storage options we have today?

• Public Cloud and Private Cloud.

• Distribute you data

Lecture 3. MySql. SQL. Introduction. How it works?

• What is MySQL?

• How Does MySQL Work?

• Why is MySQL so Popular?

Lecture 4. Installing and Launching MySQL Workbench. How to Get Started with MySQL Workbench.

• MySQL server setup

• Initial settings

• Getting Started with MySQL

Lecture 5. Practice. Excel table into MySql. Import Excel data into a MySQL database.

• Import Excel data into a MySQL

• Create a new MySQL table.

• Most Common Queries.

• SELECT, DROP, UPDATE query mysql

Lecture 6. Spark. Hadoop. Data's Distribution. A Storage System for Big Data.

• What is Hadoop?

• Spark vs MySql

• Spark. Analytics engine for big data processing

Lecture 7. Installing and Launching Apache Spark. Download and Get Started.

• Installing Apache Spark

• updating PATH environment

• Getting Started with Spark

• Launching Apache Spark

Lecture 8. Practice. Connecting Python To The Spark. Get Started with PySpark and Jupyter Notebook.

• Installing Anaconda On Windows

• Running the Jupyter Notebook

• Connecting Jupyter notebook to Spark

Lecture 9. Practice. Connecting MySQL with Spark. Export Data from Mysql to Spark. Distributed Computing with Spark.

• Connecting Jupyter notebook to Spark

• How to set up PySpark for your Jupyter Notebook

• Export Data from Mysql to Spark

• Importing Spark Dataframes from MySQL on Jupyter notebooks

The course is best-suited for learners who are interested in Big Data and Machine Learning (using Python) or for learners who already have Python programming skills but want to practice with a hands-on, real-world data project can also benefit from this course.