Learn efficient techniques for managing large datasets. From compressed file management with `zipfile` and `tarfile`, to batch processing, this course equips you to handle data at scale.
Overview
Syllabus
- Unit 1: Managing Data from Compressed Datasets
- Unveiling File Names in Zips
- Accessing and Reading Compressed Data
- Extract Top Cosmic Objects from Data
- Unzipping Secrets of JSON Data
- Unlocking Book Author Names
- Unit 2: Writing and Reading Large NumPy Arrays
- Optimizing Data Storage with NumPy
- Enhance Array Operations with NumPy
- Efficiently Store NumPy Arrays
- Unit 3: Writing Data in Batches
- Fix the Batch Writing Bug
- Write Data to CSV in Batches
- Appending Random Strings to Files
- Unit 4: Reading Data in Batches from Multiple CSV Files
- Discover the Most Expensive Car
- Streaming Data for Efficiency
- Dynamically Handling Multiple CSV Files
- Mastering Data in ZIP Archives