After completing the Yelp reviews project, I realised that I need to spend some more time in understanding theory behind the NLP process.
I have collected the resources which I think are useful. I’m sharing it publicly so that if you are also learning NLP, it could help you in some way.
The Purpose ?
Earlier when I was beginning to learn Web Development concepts on my own, I didn’t follow any curriculum. After few months I realised that I was in the middle of no where. That’s when I realised that having a curriculum – self or others – is necessary.
Also, having a curriculum helps to track your progress over a period of time. In this one month “Reading List” I aim to learn about overall NLP field and its implementation using Deep Learning. I also plan to learn about very basics of RNN and LSTM so that it will help in understanding other complex networks better. Moreover, getting deeper into a concept will help you to remember that concept very well. I learnt this during initial days of coding – some 15 years ago. Debugging the code by executing the code line-by-line and watching the variables at every step, was the best way to understand the code and the concept behind the algorithm.
And, having a deadline will certainly motivate me to focus more on the topic regularly.
I’ve mostly included articles that explain RNN/LSTMs in detail. I’ve not included anything on Transformers because to understand that, it will be helpful if basic concepts are better understood.
I’ve included a Kaggle competition as a Project. This should help in understanding NLP concepts better.
Here’s the list,
- Outline of NLP – Wikipedia article which outlines different concepts in NLP like Prerequisite Technology, Subfields, Process, History etc. This should be read once in a while to understand where we stand in terms of our understanding.
- NLP – Wikipedia article that talks more about different problems that are solved with the help of NLP
- The Unreasonable Effectiveness of Recurrent Neural Networks – Published in 2015, this post by Andrej Karpathy is still one of the best article on RNN. In this article he demonstrates what language models are capable of.
- Anyone Can Learn To Code an LSTM-RNN in Python – An Article by Andrew Trask , published almost 5 years ago, is still a good reference point for understanding and implementing RNN, just with numpy.
- An Introduction to Recurrent Neural Networks for Beginners – Another good article on implementing RNNs with just numpy. This article implements a sentiment classification task with RNN
- Implementing a RNN with Python, Numpy and Theano – This article talks about building RNN using numpy and optimising the code using Theano
- Understanding LSTM Networks – This article gives best theoretical explanation of RNN and LSTM
- Jigsaw Multilingual Toxic Comment Classification – This is a Kaggle Competition where you’re supposed to identify toxic comments across multiple languages using TPUs.
- Recurrent Neural Networks by Example in Python – A small project to implement learnings from RNN
- Sequence Models | Coursera – Very good introductory course on NLP by Andrew NG. This covers basics of language modelling, sequence modelling, voice recognition and especially it covers basics of RNN, LSTM networks in depth.
- Stanford CS224N: NLP with Deep Learning | Winter 2019 – Stanford’s course on NLP taught by Christopher Manning. I want learn this in two passes. In first pass I’ll just sift through all the concepts. In the second pass I’ll focus on specific topics of my interest.
- Advanced NLP with Spacy – A very good course on Spacy by one of the core developers of Spacy itself.
How I’m planning to Read ?
Reading and practicing for at-least two hours per day.
I’ll try to create an online repository of my work for future reference.
If you’ve got any questions, or just want to say Hi, feel free to talk to me