What we saw last time

• Course
• Projects
• State of AI
• Tools: Colab
• Demo: data analysis using Colab and Gemini

Questions ? Feedback ?

Today

• news review
• open source vs closed source
• data sources
• data formats
• working with pandas
• the web

• Project reviews

• Guided practice: data formats (json, csv) and the python pandas library

At the end of this class

You

• understand what an API is
• know the 4 operations in the REST protocol
• can edit a JSON file

the hugging face release dashboard

2024 AI Timeline - Hugginface

Major distinction between open and closed source models

Open source vs closed source

A distinction relevant for all software including AI models

Open source

Since the code is public: Transparency, security, innovation, cost-effectiveness

• Innovation & Flexibility: Users modify and enhance the software independently. Community-driven development, faster bug fixes, customization of code
• Security & Transparency: Security, there’s no virus. Issues are identified and patched quickly by the community.
• Cost-Effective: for users: free to use and modify, reduced licensing costs; no vendor lock-in. and for creators: community driven intelligence.
• Knowledge Sharing: Developers learn from existing code, accelerating skill development and innovation, shared knowledge and best practices.

Linux is open source

• Linux is an operating system (OS)
  • software that manages the computer’s hardware (CPU, memory, storage) and lets you run applications.
• Created in 1991 by Linus Torvalds as a free alternative to proprietary systems.
• the code is public (GPLv2 license), anyone can copy, improve, and share.
• Runs most (85 - 90%) of the internet
• Powers Android phones, TVs, cars, spaceships and nearly all supercomputers.
• Vendor-neutral: scales from a $5 microboard (rasberry Pi) to hyperscale datacenters.
• Cost & control: open source, huge ecosystem, no forced upgrade cycles.
• excellent security: no virus, no bugs
• exists in 200–300 active distributions (versions)

Open source LLMs

Different levels of openness:

• model : you can download the model and use it as is
• code: the code to create the model
• training data: the data used in training the model

Some models are fully open (DeepSeek), partially open (LLama, Mistral 7B), or closed (OpenAI o1, Claude Sonnet, Gemini)

If you have the weights of a model you can fine tune it on your own data. Light version of training a whole model

Closed source models: issues

Lack of Transparency & Reproducibility: architecture, weights, hyperparameters, and training data hidden

• copyright issues : on what data was the model trained. Many ongoing lawsuits.
  • see aifray an excellent resource on artificial intelligence regulation
• model bias :
  • grok controversy MechaHitler
  • hard to detect systemic biases, skewed behaviors, or harmful training artifactss

Recent open source

Open source models are catching up with closed source models

Q: What does this mean ?

Context window

• 1 token ≈ 0.75 words
• So 1,000,000 tokens ≈ 750,000 words.
• 1 page ≈ 250-300 words
• 1 million tokens ≈ 2,500–3,000 pages of a standard book.
• ~1.3× War and Peace by word count.

language variations

MoE Mixture of Experts

• Mixture of Experts (MoE) = many small specialists models + a ** router.

The router model picks the proper expert models given the input and combines their outputs.

=> Better quality (specialization) and lower cost (sparse compute).

A Visual Guide to Mixture of Experts

How experts form (training)

• Model trains end-to-end: router + all experts learn together.

• Router learns which experts fit each token/type of text.

• Experts get practice on what they see most → specialization emerges

• Load-balancing & capacity limits keep work shared so all experts improve.

Questions:

• so how do you train expert models ?
• is there a taxonomy of all the subjects ?

=> No

Data & sampling weights

• Train on a big mixed dataset (web, books, code, Q&A, dialogue, languages).

• We choose sampling weights by goals → start with an initial mix, then test & adjust.

• Guardrails: min/max quotas, quality filters, de-duplication; optional curriculum (shift weights over time).

• MoE-specific nudge: up-weight data that wakes underused experts.

=> Net effect: the router + data mix shape who does what, efficiently.

Datasets & datasources

• News and Media: Newspaper archives (NYT, …), Youtube (yt-dlp), websites
• Scholarly and Scientific Articles: Academic journals, PubMed, arXiv, JSTOR.
• Encyclopedic and Knowledge Bases: Wikipedia / Wikidata
• Entertainment and Cultural Datasets: IMDb for film and television data, GoodReads
• Government and Legislative Data: Parliamentary records, government publications, election results
• Consumer Reviews: Amazon Reviews, CellarTracker
• Social networks: Reddit, Instagram, TikTok, Facebook, Twitter, Telegram, …

Kaggle datasets

Obtain and analyze an existing CSV dataset for the project.

• NYT Articles: 2.1M+ (2000-Present) Daily Updated
• NYT Articles: Small Processed 500k Version
• IMDB Dataset: of 50K Movie Reviews
• arXiv Dataset
• GoodReads 100k books
• GoodReads Best Books

https://www.kaggle.com/datasets

Google datasets search engine

https://datasetsearch.research.google.com/

Data sources

• European data https://data.europa.eu/en
• World bank https://data.worldbank.org/
• France https://www.data.gouv.fr/ and most countries
• Paris https://opendata.paris.fr/pages/home/ and most capitals
• French energy agency https://data.ademe.fr/ and other Gov or non gov agencies

Media

• https://open-platform.theguardian.com/
• https://developer.nytimes.com/apis

rss feeds

RSS on wikipedia

RSS = “Really Simple Syndication.” A standard way for websites to publish updates.

How it works: Sites expose a feed (an XML file). Your reader/aggregator checks it and shows new posts in one place.

Why it’s nice: Chronological, no algorithms, no ads injected by platforms, and privacy-friendly (you pull info; nobody tracks your clicks by default).

What you can follow: Blogs, news sites, podcasts, newsletters, job boards, YouTube channels, forum threads.

Finding feeds: Look for the RSS icon, “/feed” or “/rss” on the site, or a “Subscribe”/“Follow via RSS” link.

ex: https://oilprice.com

Google alerts in RSS feeds

https://www.google.com/alerts](https://www.google.com/alerts)

yt-dlp

Super efficient video downloader

Can also be used for transcripts / subtitles to build a corpus

https://github.com/yt-dlp/yt-dlp

https://www.pythoncentral.io/yt-dlp-download-youtube-videos/

yt-dlp --write-subs https://www.youtube.com/watch?v=example

Example, pick a media, get all videos, download subtitles or get videos, extract images, analyse images etc

examine coverage of a topic by a media. commentary, images, etc

EU parliament debates

Beyond html: JSON

There are multiple API standards and languages

And APIs can return all sorts of content: html, text, xml, pdfs,

APIs often return raw data formatted as JSON

JSON (JavaScript Object Notation) is a lightweight data format.

• easy for humans to read and write.
• easy for machines to parse and generate.

JSON

how machine exchange data

• Universal & simple: Text-based, easy for humans to read and write.
• Language-agnostic: Supported natively in virtually every language (JSON.parse, JSON.stringify).
• Web-native: Plays perfectly with HTTP/REST, GraphQL, Web APIs, and browsers.
• Lightweight & fast: Smaller and quicker to parse than many alternatives (e.g., XML).
• Flexible structure: Handles objects, arrays, numbers, strings—good for nested data.
• Great tooling: Linters, formatters, validators, and JSON Schema for contracts.
• Interoperable storage: Used by NoSQL DBs (e.g., MongoDB), config files, logs, and events.
• Streaming-friendly: Works well for real-time APIs and message queues.

Common data types

• strings
• numbers: integers or floats
• booleans : true or false, 0 or 1. t or f
• 1 dimensional : arrays, lists, sets, … in JSON : []
• 2 dimensional : matrices, tables, dataframes … in JSON : {}
• 3+ dimensional : tensors

Missing data

• Null
• NaN
• None
• ”” : empty string

How do you edit a JSON file ?

• do not use word or google docs. These are for real human text not data
• use a code editor but it’s overkill for just editing JSON
  • vscode, windsurf
  • sublime, atom, Xcode
  • github.dev
• An online JSON formatter and editor is a good alternative
  • for instance https://jsoneditoronline.org/

csv

csv stands for : comma separated values

in 🇫🇷🇫🇷🇫🇷 France, we use ; as a separator because numbers use , as a decimal separator 🤪

• France: 123.456,78
• Rest of the world: 123,456.78

csv files with a tab separator are sometimes called tsv

A csv file is a spreadsheet.

pandas

THE library to handle data.

Loads a csv file into a dataframe

A dataframe is exactly like a spreadsheet. columns and rows

import pandas as pd

df = pd.read_cv('titanic.csv')

basic manipulations

print(df)
df.head()
df.tail()
df.columns
df.dtypes
df.shape
df.describe()
df.isnull().sum()
df.isnull().mean()
df.isnull().sum() / df.shape[0]
df.isnull().sum() / df.shape[0] * 100
df.isnull().sum() / df.shape[0] * 100

Selecting columns

df['column_name']
df[['column_name1', 'column_name2']]
df.loc[:, 'column_name1':'column_name2']
df.iloc[:, 0:2]

Selecting rows

df.loc[]
df.iloc[]

Masking

1. create a logical mask, a condition

   mask = df.age > 18
   

2. only get the rows for that condition

   df[mask]
   

3. combine masking and selection

column_names = [list of column names you want to see]
df.loc[mask, column_names]

demo

In google colab

• load the titanic dataset into a pandas dataframe
• apply these techniques to explore the dataset

Your turn

1. load imdb_top_1000.csv either from local or from the url into a pandas dataframe called df

data dictionnary is available at : https://www.kaggle.com/datasets/mayankray/imdb-top-1000-movies-dataset

Practice sheet

Projects

Let’s review your projects

fill in the google spreadsheet for projects

• team
• project title
• project description
• data source

https://docs.google.com/spreadsheets/d/1PcCnVB6wnJVXIreziA89dtqpGRGylk03sqUnTY-HYj4/edit?usp=sharing

next week

• build your dataset with APIs
• APIs
• NYT API, wikipedia API, Guardian API

new data source:

Ethan Mollick : https://www.oneusefulthing.org

Exit ticket

https://forms.gle/9eE9PUR6mFC2szq47