Challenge Problem

We ask fellows to work on a small challenge problem to assess problem solving and coding capabilities:

  1. Select a problem from the list below
  2. Perform your analysis in a well-commented Jupyter notebook and post on Github or Google Colab
  3. Share your notebook with us

Some hints for hacking our challenge:

  • Ask yourself why would they have selected this problem for the challenge? What are some gotchas in this domain I should know about?
  • What is the highest level of accuracy that others have achieved with this dataset or similar problems / datasets ?
  • What types of visualizations will help me grasp the nature of the problem / data?
  • What feature engineering might help improve the signal?
  • Which modeling techniques are good at capturing the types of relationships I see in this data?
  • Now that I have a model, how can I be sure that I didn't introduce a bug in the code? If results are too good to be true, they probably are!
  • What are some of the weaknesses of the model and and how can the model be improved with additional work?
 

Select Your Challenge Problem

 

One-shot Learning

Omniglot, the “transpose” of MNIST, with 1623 character classes, each with 20 examples. 

Use background set of 30 alphabets for training and evaluate on set of 20 alphabets. Refer to this script for sampling setup.

Report one-shot classification (20-way) results.


Image Segmentation

Apply an automatic portrait segmentation model (aka image matting) to celebrity face dataset


Language Detection

European Parliament Proceedings Parallel Corpus is a text dataset used for evaluating language detection engines. The 1.5GB corpus includes 21 languages spoken in EU.  

Create a machine learning model trained on this dataset to predict the following test set.


Global Terrorist Attacks

Global Terrorism Database (GTD) is an open-source database including information on terrorist events around the world from 1970 through 2014. Some portion of the attacks have not been attributed to a particular terrorist group.

Use attack type, weapons used, description of the attack, etc. to build a model that can predict what group may have been responsible for an incident. 


Recommender System

Build a basic recommender system based on any of the Lab41 dataset references. 


Challenge Problem Frequently Asked Questions

 

What are you looking for in challenge submissions?

1. Problem solving ability - did you understand the problem correctly, and did you take logical steps to solve it? 

2. Machine learning skills - what sort of models did you use? How rigorous was your exploratory analysis of the data, your choice and fine tuning of models, and your assessment of results.

3. Communication skills - is your solution readable and well explained? Messiness and raw code with no explanation does not reflect well on your potential for working well with our business partners during the fellowship.

 

What are some common mistakes I should avoid?

Skipping exploratory analysis and feature engineering
Do not jump straight into fitting models without demonstrating to us, in your Jupyter notebook, that you have understood and thought about the dataset.

Choosing models with no explanation
Please use the notebook to explain your thought process. We care about this as much as we care about your results.

Unreadable notebooks
Make sure to run your notebook before sharing so that we can see the results. We won't be running your code on our machines. On the flip side, please do not print out the entire dataset or endless rounds of epochs.

Overly simplistic final results
Your final results should consist of more than a single number or percentage printout. Explain why you chose the success metrics you chose, and analyze what your output means.

 

When are the challenges due?

All deadlines can be found on our apply page.

 

What is the next step after submitting my challenge problem?

After we review your challenge, candidates selected to move forward in the application process will receive an email with an invitation to schedule a 45-minute interview with a mentor or former fellow. Be prepared to discuss your challenge. You will likely be asked to explain why you chose the model(s) you used, and asked questions gauging how deeply you understand the model(s).