Strides in Reducing Noise in Data for Improved Analysis Outcome

News

Guess who’s hanging out in your backyard and why? It’s the bespectacled Professor with the pruning hook! Really? Well, not literally.

You see, your backyard is your academic (or professional) discipline and its lifeline is data, accurate data for that matter, and that’s where the challenge is, and Professors like challenges.

In Massachusetts Institute of Technology (MIT) Professor Tamara Broderick’s words, “Working in data analysis, you get to hang out in everybody’s backyard, so to speak.”  

The good Professor liberally enthuses: “You really can’t get bored because you can always be learning about some other field and thinking about how we can apply machine learning there.”  We can’t agree more.

Her pruning hook is Bayesian inference, and she uses it to quantify uncertainty and measure the robustness of data analysis techniques.” That is invaluable.

Data quality concerns are a reality. Statistical artefact, for instance, is a big concern for the validity of the findings. The data can turn out to be one big mess of weed and floral tangles in the backyard, limiting its beauty and relevance.

At C4DLab we immerse ourselves in other people’s backyards too to tease out disruptive digital ideas that can turn into opportunities for a better life.

This quest can only be as successful as it is data-driven. Yet that is only the beginning. Why? You must always have the right data to drive action.

It’s the reason we are excited by Professor Broderick’s effort and are hanging on to her question: “What do we know from data analysis”. The second question is perhaps more instructive: “How well do we know it?”

Professor Broderick believes that “we live in a noisy world, and we can’t always get exactly the data that we want.” It means we must do something.

We must “cut through” the fog of noise and see the world through the lenses of data integrity; thus there is no harm in agreeing that the key to solving problems is posing but not stopping at the question, “How do we learn from data?”  Beyond that, we must “recognize that there are limitations and deal appropriately with them.”

“I’ve always been really interested in understanding not just ‘What do we know from data analysis,’ but ‘How well do we know it?’” says Prof. Broderick, who is also a member of the Laboratory for Information and Decision Systems and the Institute for Data, Systems, and Society.

“The reality is that we live in a noisy world, and we can’t always get exactly the data that we want. How do we learn from data but at the same time recognize that there are limitations and deal appropriately with them?” she concludes.

Read the full article at: https://news.mit.edu/2024/tamara-broderick-quantifying-uncertainty-0301

Share this

Leave a Reply

Your email address will not be published. Required fields are marked *