The problem
Before you can write a research proposal, you have to know what has already been written on your topic. Who is working on it. Whether the field is alive and growing, or quietly running out of air.
Done by hand, that is forty browser tabs open at once. You copy a title into a document, then an author, then the year, then you lose the tab and open it again. By Thursday it is a mess you have lost track of, and you still cannot answer the one question that actually matters — is this field growing?
It does not scale. Twenty papers is tedious. Two hundred is impossible. And a pile of copied titles in a document cannot be counted, sorted, or charted. You have the information and still cannot see the shape of it.
This is exactly the kind of job a computer is good at and you are not.
What your program does
- You choose a topic and how many papers you want back — a few dozen, not thousands.
- Your program asks a free research website for papers on that topic.
- What comes back is data, not a webpage — you reach in and pull out the title, the authors, the year.
- You save it as a spreadsheet, count the papers per year, and draw a chart.
Four steps. You already know how to do two of them — pulling a field out of some data is just working through a dict, and counting things per year is a loop with a tally. The asking and the charting are the parts you will have to go and learn.
What you'll be holding at the end
A spreadsheet on your computer, full of real papers on your real research interest — the exact topic you plan to apply on. Not a toy dataset. Yours.
And next to it, a chart showing how the field has grown, or shrunk, over the years. A line that climbs, or a line that sags. Something you can point at in an application and say: here is the shape of the conversation I want to join.
Run it again next month, and the numbers are current again in a few seconds.
What you'll have to find out
These are the questions you take to the docs, to an AI, or to me. Nobody has taught you the answers.
- How does Python ask a website for data, rather than for a webpage? When you type a topic into a search box you get a page meant for human eyes. There is another way to ask that hands back plain data instead.
- What shape does that data come back in, and how do you dig a title or a year out of it? It will not be a flat list. It is a thing inside a thing inside a thing, and you have to find the path down to the piece you want.
- How do you read and write a big table of data — a spreadsheet — from Python, rather than printing rows to the screen?
- How do you draw a chart and save it as a picture you can drop into a document?
- One more, and it matters most: choose a source that needs no signup and no key. Some research websites are free and open — you can just ask them. Others make you register and hand you a secret code first. Find the open one. That choice will save you an evening.
Remember the recipe: say the problem in plain words, find the box, install it, find the smallest example that works, run it, check it, then use it in your own function.
Requirements
To pass, your project must:
- Fetch its papers from a real source on the internet — actually asked for and received, not a handful of titles you typed into your program by hand.
- Use at least one function you wrote, with parameters and a return value.
get_papers(topic, how_many)is the obvious one. - Save the results to a file — a spreadsheet you can open and read after the program has finished.
- Come with a README that says what it does, how to run it, one thing it can't do, and one thing you would add next.
- Be something you can explain out loud, line by line. If a line is in your program and you cannot say what it does, either find out or take it out.
Done, and going further
Done is twenty papers on one topic, fetched from the internet and saved to a file. That is a pass, and it is a real achievement — you found a way to ask a website for data, learned the shape it came back in, and dug out what you needed.
Going further, if you have the appetite:
- The chart of papers per year, saved as a picture.
- The most frequent authors — who keeps showing up in this field?
- A filter by year range, so you can look at just the last decade.
- Export a bibliography — a clean, formatted list you could paste into a proposal.
Where people get stuck
A network call can fail, and that is normal. When you ask a website for something, sometimes it does not answer — the connection drops, the site is busy, the internet is simply allowed to say no. This is not a bug in your code. Run it again. If you want to be kind to yourself, wrap the ask so a single failure does not sink the whole program.
The data comes back nested. It is a thing inside a thing inside a thing, and the title you want is three levels down. Digging into it takes patience — print what you got back, look at it, find the next layer, print again. Peel it one layer at a time until you reach the piece you want.
Asking for five thousand papers at once is rude. A free website is doing you a favour, and hammering it will get you blocked. Ask for a few dozen. That is plenty to see the shape of a field, and it keeps you welcome.
A paper with no year will break your chart. Real data is messy. Somewhere in your results is a paper with no year listed, and when your counting loop reaches it, it falls over. Decide what to do with the blanks — skip them, or bucket them as "unknown" — before they decide for you.