The Ethics of Big Data with Cathy O'Neil
Can Big Data actually hurt society? Carl and Richard talk to Cathy O'Neil about her book, Weapons of Math Destruction. Cathy has been deeply involved with machine learning and big data for decades and has a broad view of both the potential and dangers of the technology. The conversation dives into understanding how this technology amplifies bias and how that bias ultimately shapes behavior. The trick is to acknowledge that the bias exists - once you see it, it is possible to overcome its effects. Lots of great thinking from an expert in the space!
Cathy O’Neil earned a Ph.D. in math from Harvard, was a postdoc at the MIT math department, and a professor at Barnard College where she published a number of research papers in arithmetic algebraic geometry. She then switched over to the private sector, working as a quant for the hedge fund D.E. Shaw in the middle of the credit crisis, and then for RiskMetrics, a risk software company that assesses risk for the holdings of hedge funds and banks. She left finance in 2011 and started working as a data scientist in the New York start-up scene, building models that predicted people’s purchases and clicks. She wrote Doing Data Science in 2013 and launched the Lede Program in Data Journalism at Columbia in 2014. She is a regular contributor to Bloomberg View and wrote the book Weapons of Math Destruction: how big data increases inequality and threatens democracy. She recently founded ORCAA, an algorithmic auditing company.