How to conduct research effectively

Translated by 36Kr Tech Translation Team, focusing on technology, business, career, and daily life, with a spotlight on emerging innovations, fresh perspectives, and global trends from overseas.

Editor's Note: Real scientific research never means waiting for inspiration in a fancy office. It is an efficient iteration process built around curated information diets, robust engineering toolchains, and resilience against setbacks. This article is a translated piece.

Virtually no one has ever formally taught you how to conduct research. All you get is a desk, a pre-selected research topic chosen by others, and a vague instruction to produce innovative outputs. As a result, most people can only reverse-engineer what research looks like from superficial glimpses: published papers, Twitter feeds, and industry conference keynotes. In the end, they only learn how to "appear to be a researcher" rather than "how to be a real researcher". Genuine research capability is actually a collection of small, discrete skills — nearly all of which can be cultivated through deliberate practice.

Find Your Own Research Topics

Richard Hamming had a habit at Bell Labs that made him an unpopular figure at lunch tables. He would ask anyone sitting nearby what the most important problems in their field were, and then follow up with: "Why aren't you working on those problems?" People would immediately get up and switch to other tables. This question stings because most of us cannot come up with a decent answer. We do not select research topics actively; we passively absorb them — from our advisors, from research released by top labs last quarter, or from a viral paper that everyone is retweeting and commenting on this week.

The trouble with passively adopting topics is that you only remember the final conclusions, without understanding the underlying reasoning process. You know a well-known lab is focusing on a certain direction, but you have no idea why they care about it, what results they expect to find, or what circumstances would make them abandon that line of work. When they shift their research focus, you might only realize it a full year later. Worse still, on a topic that has already become a hot trend, you are racing against thousands of people who started earlier than you and have far more computing resources at their disposal.

John Schulman, in his *Research Guide for Machine Learning*, divides research work into two modes. The first mode is reading literature to find incremental gaps to improve. The second mode is choosing a result you deeply wish existed, then working backward to design the experiments needed to reach it. He strongly advocates for the second mode, because it is the path to true originality. A goal you genuinely care about will lead you into uncharted territory that no review paper has ever explored.

Meanwhile, academic taste is often talked about as if it were an innate talent. But it is more like a muscle that can be trained. Before running every experiment, predict what its result will be. Cover up the results section of a paper and guess what the data looks like only based on its methodology. Track which of this year's published work you believe will still matter in two years, and later check your prediction accuracy. Repeat the "predict and correct" cycle hundreds of times — that is how every good model is trained, including the one running in your head.

Upgrade Your Information Inputs

Shared reading lists only lead to homogeneous thinking. If your entire information diet consists of the arXiv trending page and content filtered through group chat threads, you will inevitably reach the exact same conclusions as everyone else at the exact same time. Which, of course, means those conclusions are almost entirely worthless.

The value of classic, old research papers is drastically underrated. This field constantly replays its own past with a delay: Mixture of Experts (MoE) dates back to 1991, LSTM was born in 1997, and backpropagation became mainstream in 1986. Rich Sutton wrote *The Bitter Lesson* in 2019 in roughly 1,000 words, and its predictions for the future of the field are far more accurate than review papers 10 times its length. Claude Shannon gave a 1952 lecture on creative thinking, where he explained his core trick: reduce a problem to an almost trivial size, solve that simplified version, then add the complexities back one by one. That single technique alone will break down far more barriers than any modern "productivity hack".

Breadth of knowledge is just as important as depth. Research in interpretability draws heavily from neuroscience. Evaluation design is essentially mechanism design wearing a lab coat. Understanding exactly how a GPU accesses memory will let you know which architecture papers are doomed to fail, long before benchmark results come out. In machine learning, solid, rigorous statistics is perhaps the rarest skill of all — many published "rigorous results" are little more than educated guesses with error bars attached.

One more thing: read the actual full paper, not just the Twitter thread summarizing it. The appendix is where the embarrassing secrets (or critical flaws) are hidden, and the "Limitations" section is often the most honest part of the entire publication.

Document Everything

Paul Graham once pointed out that an idea often feels fully formed before you write it down. But putting it on paper will catch the gaps your mind has quietly hidden: assumptions you never tested, reasoning steps that do not actually hold up, and two claims that are secretly contradictory.

Feynman's principle was that the first person you must avoid fooling is yourself — because you are the easiest person for you to fool. Writing is the cheapest defense mechanism humanity has ever invented. Darwin took this even further, turning the principle into a formal process: he wrote down every fact that contradicted his theory immediately, because he noticed his brain was far faster at erasing unfavorable evidence than it was at retaining favorable evidence. Your memory does the exact same thing with failed experimental runs. Keep a lab notebook: document your hypothesis, setup, expected results, actual outcomes, and updated takeaways. Rereading last month's notes will bring you a humbling clarity that no peer reviewer can ever match.

Then, make parts of that work public. In their essay on "research debt", Chris Olah and Shan Carter noted that many fields stagnate because they cannot process the sheer volume of new ideas — and clear, accessible explanations are real, tangible contributions, not trivial "service work". Many people working on interpretability today found their path through easy-to-understand blog posts, not conference papers. A public body of written work is also your strongest credential, because it cannot be faked: it shows exactly how you think.

Tighten Your Iteration Loop

Alec Radford's success stories are almost never about a single brilliant "aha!" moment. They are all about volume: running more experiments every day, discarding more wrong ideas every week, and updating your model of reality faster than anyone else. That is the real game. The speed of research is largely determined by how quickly you can find out you were wrong.

This makes building a smooth engineering toolchain one of the most critical research activities. Launching an experiment should take one single command. Generating a plot should take another. Every experiment should be fully reproducible from its configuration, and comparing two runs should take seconds, not an entire afternoon of code archeology. Andrej Karpathy shared a trick in his neural network training guide that gives 100x return on effort: overfit a single batch of data before scaling up to full training. In 30 seconds, you eliminate half of all bugs. Shrink everything down to a low-cost scale, get it working perfectly, then go burn through compute resources.

At the same time, stop treating engineering as a sidekick to research. At the cutting edge, the two are inseparable. Only researchers who can build their own test frameworks, evaluation systems, and data pipelines can actually get their hypotheses tested. Everyone else is stuck waiting in line.

Keep Your Eyes on Real Outputs

A steadily decreasing loss curve is not analysis — it is a placebo. Your experiments produce far more information than you ever consume: text logs, failed cases, outliers in the long-tail distribution. Most of that data sits quietly in log folders, never to be looked at again.

Karpathy's methodology even includes a step before writing any training code: spend hours manually going through raw data. Most bugs in machine learning hide in the data, and they are "silent failures". The program will not crash — you will just get a mediocre model, and a completely wrong theory for why it is mediocre.

For over a decade, Andrew Ng has been teaching this unglamorous trick, because nothing works better: pull 10,000 failed cases, look at all of them, group them into clusters, and focus your effort on fixing the largest cluster. This works for models, and it works for evaluation systems too. If you have never read the actual text samples from a benchmark, you do not truly understand it. Looking at samples that show truly bizarre behavior will teach you far more than squeezing another 0.1% accuracy point out of your model.

Wander With Purpose

The first subfield you step into is usually just a coincidence of timing — so treat it that way. Before you commit to a long-term research direction, put in real effort to learn about interpretability, evaluation, reinforcement learning, and systems architecture. Somewhere in this field, there is a niche where your unique, quirky strengths turn into an unfair advantage — and the only way to find it is to pay tuition in multiple different areas. No one gets to skip that step.

Run a quick, throwaway prototype for every new idea, and let most of them die early. Tune your baselines aggressively, because the graveyard of machine learning is full of "performance improvements" that vanished the second they were tested against a well-tuned baseline — and peer reviewers are the worst possible people to teach you that lesson. Run ablation studies repeatedly until you know exactly which single component is driving the result. Usually there is only one, and it is almost never the one mentioned in the paper's title.

Breadth is also an insurance policy. Every subfield eventually gets saturated, usually right after it hits peak viral popularity on Twitter. The people who keep producing high-quality work through industry shifts are the ones who already know the neighboring subfields inside out.

Find Your Tribe

Hamming found a pattern that determines who ends up doing truly important work. Colleagues who kept their office doors closed might finish more work in a single year, but the ones who kept their doors open produced work with lasting impact. Because those "interruptions" bring you information about what the world actually needs. Today, your open door is probably your inbox. Keep it open.

In research, the compound interest of generosity is unmatched. Reproduce a result and share what you find; release tools you built yourself; explain complex concepts in plain language. The rewards will come months later in unexpected ways: a collaboration, a citation, or a job opportunity you never would have been able to apply for otherwise. Share your half-baked ideas publicly too — the cost of being wrong on a Twitter timeline is far lower than the cost of being wrong in a peer-reviewed published paper. And a collaborator who can tell you "this idea is bad" before you spend three months working on it is worth more than unlimited compute resources. Those relationships cannot be bought — you have to earn them.

Play the Long Game

Pasteur once said that fortune favors the prepared mind. Hamming built an entire career philosophy on that idea: knowledge and productivity compound like interest. Individually, the tiny advantages you build every day seem trivial — what you read, what you document, how fast your iteration loop runs, who you debate ideas with. But give them a few years, and they will build a career that looks to outsiders like pure luck. Start building that compound interest long before you feel you need it. Your future self already knows that putting in that effort now is the cheapest possible way forward.

Translated by boxi.