Programming Language II (Data Engineering with Python)

Course objective and learning outcomes

The objective of this course is to acquaint departmental students with the fundamentals of Data Engineering through the Python programming language. Following a concise introduction to programming methodologies, students will gain hands-on experience in extracting and storing substantial volumes of environmental data, as well as processing and visualizing this data in real-time. They will also learn problem-solving and debugging strategies. Concurrently, students will learn to interface with relevant tools, sensors, and electronic measurement devices using Python. Students will showcase their Python skills through practical programming tasks and exercises. These will not only validate their learning but also contribute significantly to their portfolio.

Course Content

1. Introduction, Variables, Expressions and Statements, Conditional Execution, Data Structures
2. Functions, Loops and Iterations
3. Strings and Data File Handling
4. Lists, Dictionaries and Tuples
5. Regular Expressions – REGEX
6. Downloading and transferring data over the internet
7. Use of web services and Application Programming Interfaces
8. Introduction to Databases
9. Anaconda Virtual Environments, Jupyter Notebooks
10. Data Analysis – The NumPy and Pandas Libraries
11. Data Visualization – The Matplotlib Library
12. Handling messy and missing data
13. Sampling and understanding of experimental data

Student Evaluation

The successful completion of the course relies on four progressively challenging programming tasks. The initial two tasks involve exercises on concise programming methodologies. The third task deals with the efficient management of large amounts of unorganized and incomplete data. The fourth and final task is a comprehensive ETL project that incorporates all the knowledge acquired throughout the course. Additionally, the Student Assessment Process may include an oral examination.

Recommended Literature

Freely accessible online course notes:   Data Engineering with Python (15ΖΥ2Ν-Κ1)

Bibliography in Greek

1. “Εισαγωγή στον Υπολογισμό και τον Προγραμματισμό με την Python, 3η έκδοση/2022”, Συγγραφείς: Guttag John V., ISBN: 9789604911592
2. “Εισαγωγή στην Python για τις Επιστήμες Υπολογιστών και Δεδομένων”, Έκδοση: 1η Εκδ./2021, Συγγραφείς: Harvey M. Deitel, Paul J. Deitel, ISBN: 9789605127442

Bibliography in English

1. “Introduction to Computation and Programming Using Python, 3rd edition”, Authors: Guttag John V., ISBN: 9780262542364
2. “Intro to Python for Computer Science and Data Science, 1st edition”, Authors: Harvey M. Deitel, Paul J. Deitel, ISBN: 9780135404676