Q. Ethan McCallum.
Bad Data Handbook.
O'Reilly. 2013
rating : 4 : passes the time
review : 10 January 2025
Prom cranky storage to poor representation to misguided policy, there are many paths to bad data. Bottom line? Bad data is data that gets in the way. This book explains effective ways to get around it.
Among the many topics covered, you’ll discover how to:
• Test drive your data to see if it’s ready for analysis
• Work spreadsheet data into a usable form
• Handle encoding problems that lurk in text data
• Develop a successful web-scraping effort
• Use NLP tools to reveal the real sentiment of online reviews
• Address cloud computing issues that can impact your analysis effort
• Avoid policies that create data analysis roadblocks
• Take a systematic approach to data quality analysis
Much computing involves significant data processing. Before you start processing, however, you need to gather, clean, and validate your inputs. The aim of this slim book is to highlight potential pitfalls, and suggest good techniques to avoid them.
This overall discussion is very choppy, because we have a series of short chapters, each discussing a single aspect. Some chapters are a little superficial; some a little off topic; others of interest are (necessarily given the format) too brief. However, if this is approached as a form of consciousness raising, rather than a handbook of solutions, there is enough here to inform, alarm and warn many data analysts.