Labor Day Reading and Academic Year Kick-Off
Deep Learning / Machine Learning Reading and Study Guide:
Several of you have been asking for guided reading lists. This makes sense.
Your Starting Point for Neural Networks, Deep Learning, and Machine Learning
Your study program (reading and code) depends on where you are.
- Starting out (High-grass country; St. Louis to Alcove Springs): Basic neural networks and deep learning; architecture for common networks, such as CNNs (convolutional neural networks); learning rules and architecture design.
- Well on the way (Short-grass Great Plains; Alcove Springs to Fort Laramie): Deeper understanding of all the fundamental architectures and starting to understand the energy-based models.
- High-level topics (The Rocky Mountains; Fort Laramie to Fort Vancouver): The really tough mathematical / abstract materials.
The Biggest Challenge
There are challenges at every point of this journey.
Perils along the way caused many would-be emigrants to turn back … Nearly one in ten who set off on the Oregon Trail did not survive. Oregon-California Trails Association
The biggest danger in the whole trek is getting caught in a Donner’s Pass. This means: getting lost in the high-altitude topics, not knowing which direction will get you out safely, and running out of resources – time, energy, money.
My focus is on getting you through the Rockies; through the most dangerous high-altitude topics. (That’s why we’ve got the Seven Essential Equations; that’s a guide through the mountains.) We’re going to know our route, and we’ll be expeditious; we’ll avoid getting trapped in the passes. This is why I’m working on my Cribsheet; it’s going to be a legend to go with the Seven Equations map.
Annotated Readings-and-Resources Lists
This weekend, though, I’m organizing some reading materials – Readings and Resources. They include YouTube vids and short courses. My lists will be annotated, so you’ll get a sense of what to do first, second, and third.
- The Armchair Traveler: You want to read ABOUT neural networks, deep learning, and machine learning – but are not ready to do the equations and the code. Or, you’re just looking for something to download and listen to while you take the kids on a short (not 2,000-mile) roadtrip. Go for survey-level books and articles.
- Go to No-Equations AI Reading (and science fiction bonus link): AI reading that you can listen to while you drive – or read while you’re going to sleep. Some of these books (maybe all) are available on iTunes as podcasts.
- A few applications-oriented survey articles: deep learning and AI surveys and overviews, including industry trends,
- Something really new, every 32 years: A brief history of AI and neural networks, and why we’re ready for something new, right now: a brief history; 64 years and two eras of AI and neural networks.
- Starting Out: What I would read if I were starting today
- Great Plains (arid and dry, just working through it): Deep learning – (slightly) more advanced.
- Rocky Mountains (the challenging math): Energy-based methods in deep learning.
- Looking over my shoulder: Dr. A.J.’s personal reading list
9 thoughts on “Labor Day Reading and Academic Year Kick-Off”
Thanks for posting! For anyone interested in a great ‘beginner through to intermediate’ course, the Coursera course by Andrew Ng has been fantastic. I’m only about 3/4 of the way through the first class (out of five in the cert program), but it’s a very good introduction that covers both the coding aspect (in python and tensor flow) as well as the calculus and matrix multiplication foundations, in a way that is accessible to someone with out a very math-intensive background.
Hey, Matt – thanks for chiming in right away, and we’re on the same page here – I was putting in the link to Andy Ng’s video and looking for his Stanford materials, as well as a link to Yann LeCun’s full course materials as you were posting this – thank you!
I’m going to find his basic introductory YouTube vid next; lots of people have been praising that one.
Congrats on pushing through in your self-study!
Dr. Maren – This is completely new to me. Yet I am intrigued with your overland journey metaphor and what to read first so as to not get snagged upstream. Looking forward to the “light” reading. Thanks. Kim M.
You’re most welcome, Kim, and thank you!
There’s a lot in that “light reading” section that is very readable, and the industry news reports (starting over a year ago) are why so many people are getting into data science & machine learning; it’s the new “gold rush.”
I almost got to Fort Laramie, but then I died of dysentery.
Hey, Topher – good to hear from you, and … in your newly-incarnated life (since you didn’t quite make it to being a 49’er the last time) – are you going to be a Silicon Valley maven?
Nothing like trying the second, and third, time around!
Thanks for posting this Prof Maren. Looking forward to the course next week.
May I also add: http://course.fast.ai/index.html , which I’ve just started working through. Seems to be a good way to learn python and deep learning together.
Thanks Rahul, this is a great choice!
Fabulous to get your hands on Python and deep learning methods at the same time. Starting — VERY SOON — you’ll have your hands on my code also; will greatly appreciate your feedback – and kudos to you for getting an early start with fast.ai!