Explore the essential role of web crawling and scraping in meeting the growing data demands for both generative and traditional AI in this 32-minute conference talk from DSC EUROPE 24. Domagoj Maric provides a comprehensive examination of methodologies and tools for effective web data extraction, with special emphasis on ethical and legal considerations. Learn best practices for polite and efficient crawling while maintaining compliance with privacy and intellectual property regulations. Discover how businesses and developers can responsibly navigate the complexities of utilizing web data for AI applications. This presentation was delivered on November 22nd at the Data Science Conference in Belgrade.