Recommended Framework | |
|
I came across a framework that helps abstract some of the work with Theano. It was developed with a focus on enabling fast experimentation - enabling you to go from idea to result with the least possible delay. This is the key to doing good research. Check out: http://keras.io/getting-started/sequential-model-guide and thanks Ayal and Nitzan for drawing my attention to it. |
| פורסם ב-16/6/2016, 10:18:17 Created on 16/6/2016, 10:18:17 Создано16/6/2016, 10:18:17 تم النشر ب-16/6/2016, 10:18:17 |
Recommended Lecture: Understanding Word Embeddings (337 Taub on 16\6) | |
|
The talk will be on Thursday 16/06/2016 15:00 room 337-8 Taub Bld. The speaker is Omer Levy who is a mentor for some of the research projects and I recommend you attend and correspond with him to schedule a f2f meeting. Abstract : Neural word embeddings, such as word2vec (Mikolov et al., 2013), have become increasingly popular in both academic and industrial NLP. These methods attempt to capture the semantic meanings of words by processing huge unlabeled corpora with methods inspired by neural networks and the recent onset of Deep Learning. The result is a vectorial representation of every word in a low-dimensional continuous space. These word vectors exhibit interesting arithmetic properties (e.g. king - man + woman = queen) (Mikolov et al., 2013), and seemingly outperform traditional vector-space models of meaning inspired by Harris's Distributional Hypothesis (Baroni et al., 2014). Our work attempts to demystify word embeddings, and understand what makes them so much better than traditional methods at capturing semantic properties. Our main result shows that state-of-the-art word embeddings are actually "more of the same". In particular, we show that skip-grams with negative sampling, the latest algorithm in word2vec, is implicitly factorizing a word-context PMI matrix, which has been thoroughly used and studied in the NLP community for the past 20 years. We also identify that the root of word2vec's perceived superiority can be attributed to a collection of hyperparameter settings. While these hyperparameters were thought to be unique to neural-network-inspired embedding methods, we show that they can, in fact, be ported to traditional distributional methods, significantly improving their performance. Among our qualitative results is a method for interpreting these seemingly-opaque word-vectors, and the answer to why king - man + woman = queen. |
| פורסם ב-26/5/2016, 13:37:34 Created on 26/5/2016, 13:37:34 Создано26/5/2016, 13:37:34 تم النشر ب-26/5/2016, 13:37:34 |
Research Milestone Submission date: 19\6 | |
|
Notice this time is a hard deadline. A template for your research milestone have been uploaded. You should have your first research result by this date proving that your model is superior to the state of the art. If you still did not discuss evaluation with me or your mentor - hurry up! The remaining time for your research paper will be spent on additional experiments and academic writeup. Notice that not all of you have responded to the feedback sent to their research proposals. Research proposals who have not been approved will not be accepted. |
| פורסם ב-18/5/2016, 14:02:20 Created on 18/5/2016, 14:02:20 Создано18/5/2016, 14:02:20 تم النشر ب-18/5/2016, 14:02:20 |
HW3 is published | |
|
Submission in pairs by 07/06. This one is supposed to be short and is very useful for most of your projects. The reason it is short is to let you focus on your research projects. Good luck! |
| פורסם ב-18/5/2016, 12:33:22 Created on 18/5/2016, 12:33:22 Создано18/5/2016, 12:33:22 تم النشر ب-18/5/2016, 12:33:22 |
The 2016 Israel Seminar of Computational Linguistics | |
|
ISCOL: http://ie.technion.ac.il/~roiri/ISCOL2016 will be held on Tuesday, May 31, 2016 at the Technion, Israel Institute of Technology. I encourage all of you attend. The conference usually presents papers that were accepted at NAACL/ACL and it's a good opportunity to chat with other students in the field. |
| פורסם ב-4/5/2016, 09:41:08 Created on 4/5/2016, 09:41:08 Создано4/5/2016, 09:41:08 تم النشر ب-4/5/2016, 09:41:08 |
Class on May 8th is canceled | |
|
Due to an urgent business trip the class on Sunday is canceled. Our next lecture will be on May 15th. We will hear a lecture from Prof. Mark Silberstein about Efficient implementation of Deep learning on GPU. |
| פורסם ב-3/5/2016, 12:37:18 Created on 3/5/2016, 12:37:18 Создано3/5/2016, 12:37:18 تم النشر ب-3/5/2016, 12:37:18 |
Research proposals | |
| Please add personal emails to the submissions so feedback can be sent. |
| פורסם ב-1/5/2016, 09:36:22 Created on 1/5/2016, 09:36:22 Создано1/5/2016, 09:36:22 تم النشر ب-1/5/2016, 09:36:22 |
HW2 is published | |
|
Submission in pairs by 15/5. Good luck! |
| פורסם ב-13/4/2016, 13:41:11 Created on 13/4/2016, 13:41:11 Создано13/4/2016, 13:41:11 تم النشر ب-13/4/2016, 13:41:11 |
Microsoft Azure Free Computing Environment for our Course | |
|
Hello all, Great news! We just received a grant from Microsoft for $100/month Student Passes for 6 months for every student in class to run Azure machines for your projects. Please contact me personally to get your passcode. These codes can be redeemed at www.microsoftazurepass.com. Thanks Lotem for the idea! |
| עדכון אחרון ב-13/4/2016, 17:35:09 Last updated on 13/4/2016, 17:35:09 Последняя модификация13/4/2016, 17:35:09 تمت الحتلنة الأخيرة ب-13/4/2016, 17:35:09 |
Suggested libraries + State of the art in all NLP tasks | |
|
I added a few links to deep-learning libraries by different programming languages and a list of the state of the art in all NLP tasks and the standard datasets for each one of the tasks for you to consider for your projects. |
| עדכון אחרון ב-11/4/2016, 21:34:58 Last updated on 11/4/2016, 21:34:58 Последняя модификация11/4/2016, 21:34:58 تمت الحتلنة الأخيرة ب-11/4/2016, 21:34:58 |
Possible datasets for final project | |
|
Yahoo opens Webscope 13+TB database to public: http://webscope.sandbox.yahoo.com/ Those are quite big datasets that you might find useful for your final projects. |
| פורסם ב-22/3/2016, 11:46:52 Created on 22/3/2016, 11:46:52 Создано22/3/2016, 11:46:52 تم النشر ب-22/3/2016, 11:46:52 |
HW1 is published | |
|
Submission in pairs by 16/4. The request of submitting in pairs is crucial as the work will be planned for 2+ people. Use the "Find partner" link to find a partner. Additionally, as we had a hash collision with the course number, to get the HW do the following: Use the "Homework" tab in the GR site, instead of the "Assignments" tab for the specific course page. Good luck! |
| עדכון אחרון ב-23/3/2016, 09:40:18 Last updated on 23/3/2016, 09:40:18 Последняя модификация23/3/2016, 09:40:18 تمت الحتلنة الأخيرة ب-23/3/2016, 09:40:18 |