What Recruiters Can Learn from Moneyball


Michael Lewis’ 2003 book, Moneyball: The Art of Winning an Unfair Game, chronicles the true story of the 2002 Oakland Athletics baseball team, who, despite being at a serious economic disadvantage, were able to compete with a big market powerhouse like the NY Yankees. Their general manager, Billy Beane, made it work by using an analytical and evidence-based approach.

For most of its history, baseball was considered an unfair game. Without a salary cap to create parity, big market teams dominated the competition because they could hook and sign all of the top talent. They had the big bucks, so they fielded the best teams and won consistently.

Everything changed when Beane realized that baseball insiders (scouts, coaches, and general managers) were using an antiquated and ultimately flawed approach of assessing the best talent and the best teams. Despite having access to new, advanced metrics to benchmark talent, they continued to rely on subjective opinion to gauge player ability.

Moneyball was adapted into a film in 2011. In one powerful scene, GM Billy Beane, played by Brad Pitt, sits in the Athletics war room with his team of scouts and assistants. He proposes a radical new strategy to sign players that the league thinks are washed up or otherwise deficient in some way. Every scout in the room is shocked by his audacity, but he shuts them all down again and again with one phrase: “He gets on base.”

Beane didn’t rely on conjecture to pick the ideal talent. He didn’t let sentiment cloud his judgement and limit his ability to make the right calls. He signed people who could do their job — who could get on base — and he fielded a team that won 103 games, topping the American League West. Despite only having a third of the money as the Yankees to spend on players, the A’s won just as many games in the regular season.

This same kind of efficient, analytical approach employed by Beane can be just as effective for recruiters in business as in baseball. Historically, companies have had a problem assessing the ability of their new hires. New employees are leaving at a higher rate than ever, often because they simply aren’t suited for the role they’ve been asked to fill.

Like the scouts in Moneyball, recruiters rely too heavily on proxies for performance: the college or university you attended, your GPA or letters of recommendation. The problem is exacerbated by subjective tools such as intuition, and flawed systems such as interviews to assess potential hires. Employees aren’t judged properly, they leave, and companies spend thousands of dollars replacing them and funneling new candidates through an antiquated hiring process.

Jason Dana recently wrote a piece for the New York Times where he stressed that job interviews are essentially useless. He makes the core point that “interviewers typically form strong but unwarranted impressions about interviewees, often revealing more about themselves than the candidates.” Recruiters listen to the way that candidates answer questions, and they draw their own, often false conclusions about what the answer says about the person.

To make matters worse, candidates often misrepresent themselves in interviews. They don’t speak truly about themselves; they say what they need to say to get hired. Recruiters believe they can use intuition to suss out these discrepancies, but it’s a subjective, flawed, and often unsuccessful approach.

Unilever has radically changed its hiring practices by ditching time-consuming and costly strategies like visiting college campuses and subjecting candidates to round after round of interviews. Instead, they use algorithms to sort and recommend the best possible candidates from the pool. Unilever has found that with this process, 80% of the people who make it to the final round get a job offer.

Authess uses the same kind of AI-driven, analytical approach to gauge the best possible hires. Rather than relying on intuition, recruiters can authentically assess whether or not their candidates are right for the job. By pre-screening an applicant’s ability to organize information, solve problems, and communicate effectively on an authentic, real-world task, recruiters can ensure that every candidate they consider is capable of handling the demands of the job.

Authentic assessment is affordable, scalable, and has proven to be a better predictor of ability than the traditional, subjective approach. You don’t need the money of the New York Yankees to find, hire, and keep the people who can get on base.

In that same scene where Billy Beane proposes his new strategy to make the Oakland A’s the best possible team they can be, he is confronted by one of his scouts who just can’t believe that this analytical approach could work. Billy only has this to say to him: “Adapt or die.” We have the advanced metrics and the tools to measure an individual’s skills and abilities. It’s time to adapt or face the consequences of sticking to an old and failing model.

If you’d like to see how Authess helps you field the best possible team, request a demo.

Addressing Onboarding Shortcomings


The turnover problem that many companies face starts with how they onboard new employees. Data-driven assessment can help encourage the development growth that is essential to effective onboarding.

What is Onboarding Really For?

In a 2014 survey with BambooHR, one-third of approximately 1,000 people surveyed said that they quit their last job before they hit the sixth-month mark. Approximately 17 percent of respondents said that they left between the first week and the third month of working their new job. This costly statistic is often the result of one or two factors: inadequacy in the assessment of potential hires, and/or systemic issues in the way that new employees are brought onboard. We’ve already talked about how authentic assessment helps smooth out the hiring process.  Now we’d like to turn a critical eye toward the onboarding process and how we could do better.

To begin, it’s worth considering what onboarding intends to accomplish in an ideal scenario. Effective onboarding should do two things: (1) prepare the hire to take on their new responsibilities and (2) integrate them into the culture of the company. An early exit may indicate that the onboarding process has failed to do at least one or both.

It’s important to remember that onboarding is about much more than just training. While training plays a critical role in onboarding, the culture component is just as important.  When people enter a new company, they want to know how they will fit in. As seen with many Millennials, when new hires aren’t able to develop a sense of purpose, or they feel isolated from the rest of the team, they are much less likely to be productive and happy with their work. At the end of the onboarding process, the new employee should be able to say confidently that they are an integral part of the team and have a definitive understanding of their role to guide their growth and development.

Why It Matters

Turnover can get in the way of a business’s growth and success.  The financial cost of losing and replacing a new hire can range from 25 percent to 200 percent of their annual salary while high turnover can further be profoundly damaging to employee morale. It’s hard to feel like you are part of a team when the composition of that team is in a constant state of flux.

Onboarding may be the antidote to a high turnover rate amongst new hires. A 2013 report by the Aberdeen Group found that 91% of employees stayed through their first year in companies with best-in-class onboarding programs, illustrating the link between an engaging, effective onboarding process and a reduction in turnover.

More than just turnover, quality onboarding has also shown to have a direct impact on profit growth. It’s not hard to see why. Employees who are genuinely enthusiastic about their job do better work, and that quality of work translates to greater success for the company as a whole.

How to Make Your Onboarding More Effective

Many companies over think onboarding. You don’t have to spend thousands of dollars on a gauntlet of team-building activities to get people excited about working for you. While those things might be a nice bonus to help build team experience, the most important thing is to clarify the new hire’s purpose from day one.

New hires should have clear goals and at least one obvious thing to accomplish each and every day. Employees get off track when they are either given too little or too much to do when they start out. Effective onboarding is all about steadily ramping up responsibility as employees get more comfortable. Not only will starting small help them learn their job, but a clear focus better equips them to solve complex problems on their own.

One way to position new hires for success, and to help them understand what is expected of them, is to have them work through a standard role-based task (a typical authentic workplace problem relevant to the corporation and the new hires’ job role). This is exactly the kind of exercise that the Authess platform is equipped to provide companies struggling with their onboarding.

Because Authess captures problem-solving, approaches and strategies in addition to the final answer, Authess’ Insight reports can show new hires how their performance compares with that of top performing employees. This reporting can capture culture, behavior, ethical decision-making, creativity and other hard-to-measure characteristics that are essential to a successful onboarding process.

Above all, the most important thing you can do is give valuable feedback. It’s an obvious thing to say in concept, but much harder to implement in practice, especially in terms of the employee to manager relationship. Managers are busy with their own jobs and responsibilities and being asked to foster a new employee often leads only to stress and frustration.  

Assigning critical path work with a deadline to an employee, having to evaluate that work, then explain to that employee how their work was excellent, satisfactory or deficient, all while providing feedback that is actionable and constructive, can be extremely time-consuming.  Often, the manager is too busy to be this involved, so the new hire gets frustrated, the manager gets frustrated, and no corrective action is suggested. In other cases, the manager may be ill-equipped to evaluate the work and give constructive feedback. Inevitably, the new hire quits and goes somewhere else.

The idea behind integrating authentic assessment into the onboarding process is to provide a  low-risk and low-stress opportunity to solve the kinds of problems that the company is actually dealing with on a day-to-day basis.  With Authess, new hires get actionable feedback and a development path without burdening the manager or having another person on the payroll completely devoted to learning and development. Authess’ version of authentic assessments would take no time for the manager to administer and limited time for the new hire to complete, and the reports are auto-generated so the new hire gets feedback without burdening the productive capacity of the management team.

These tenets of feedback, growth, and problem solving are essential to why we built Authess. We’ve created a framework to help make this process of onboarding easy with a clearly articulated path and data-driven feedback to help new hires grow into their role. If onboarding is a persistent issue for your company, consider giving Authess a try.

How AI is Affecting Modern Education – Will Your Child’s Future Teacher Be a Robot?


The generation starting college this fall doesn’t remember a time before computers and the Internet. Their children may not remember a time before Artificial Intelligence.

Raised on technology, Millennials were the first generation to rely (at least partially) on the Internet to get their homework or college coursework completed. As technology continues to permeate the classroom, it’s only a matter of time before Artificial Intelligence becomes a key component in education. The opportunities presented by this development will be a decided departure in the way students learn and teachers teach...a modern twist on the timeless values of teaching.

Small Scale Disruption: Innovations in K12

If you’re familiar with Alt School, you’ll know that the future of modern education isn’t on the horizon – it’s already here. Alt School was developed by a former Google employee, Max Ventilla, who decided that algorithms driven by real-time data could lead to hyper-personalized learning for school children. This, in turn, would facilitate students to progress quickly through lessons on subjects they excelled at, and more slowly for those they found more challenging. Students are supervised by teachers, and their curricula are driven by the data fed into computer systems.  Alt School is just the tip of the iceberg in the move toward a more modern style of education.  Like many revolutionary approaches, it’s a small prototype and in its current form too expensive an approach for many school systems to implement.  

Disruptive Innovation at Scale: In Higher Ed

As Massive Online Open Courses (MOOCs) emerged in 2012 they were praised for allowing accessibility for those who couldn’t afford a traditional education, or those in remote areas who could not be physically present to attend classes. A scalable method of distributing information online, MOOCs are merely the start of the technological revolution in education.  

As they mature, these online approaches have met some resistance from critics of their comparative quality.  But, MOOCS and online courses fit the classic definition of disruptive innovation. Clayton Christensen of HBR says that disruptive innovations “are initially considered inferior by most of an incumbent’s customers. Typically, customers are not willing to switch to the new offering merely because it is less expensive. Instead, they wait until its quality rises enough to satisfy them. Once that’s happened, they adopt the new product and happily accept its lower price.” The key to increasing quality without blowing up the MOOC cost model?  High quality, scalable assessments that are driven by performance data.

And the opportunity that the explosion of online courses at traditional universities brings?  MOOCs serve as a precursor to AI and authentic assessments, by paving the way for students, educators and employers as they adjust to learning online in an environment where a massive amount of digital activity can be captured and analyzed.

Screen Shot 2017-09-08 at 11.09.03 AM.png

The Artificial Intelligence (AI) methods that Authess uses to measure complex problem-solving, are well-positioned to take a central role in higher education in the 21st century as MOOCs and online courses continue to take market share. The key to scaling these solutions while increasing quality is to make them affordable, efficient, and reliable. As the path of disruptive innovation continues, AI-driven personalized learning will be a true game-changer in online higher education.

Authentic Assessments and AI: Much-Needed Change

Benjamin Bloom’s “The 2-Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring” identified that 1-to-1 tutoring works best. MOOCs have delivered on the promise of reducing the cost of delivering instructional material, but they fall far short when trying to provide personalized learning and 1-to-1 tutoring experiences. The Holy Grail of online education is not just low-cost delivery, but truly personalized experiences with meaningful feedback and assessment capable of measuring critical thinking and problem-solving skills at scale. New developments in AI are opening the door to that kind of disruptive opportunity. Students can finally receive specific, actionable, and relevant feedback without a personal instructor sitting next to them every step of the way.

Data-driven technology takes things one step further by analyzing data from individual student performances and aggregating it, revealing trends and offering comparisons and benchmarks. This closed loop method benefits students by focusing their learning at a pace they’re comfortable with. It’s a dramatically more efficient system, too. Neither the student nor the educator waste time on what they’re already good at, but focus on where they are weak and need to up-skill.

Interacting with the course material and knowing when answers are correct (or not), is key to acquiring knowledge. It has been proven that learning is easier and better when students employ scenario-based approaches. Authentic assessments allow students to apply their knowledge to real-life scenarios. This not only reveals how well they have retained new knowledge, but also better prepares them for the workplace as they develop the skills to think critically and apply their knowledge in the real world.

Deploying these real-life scenarios at scale hasn’t historically been feasible, so we’ve fallen back on multiple choice and contrived short answer questions that only measure knowledge. These tests aren’t good predictors of students’ ability or true mastery of the concepts they are learning – just because students know factual material does not mean they know how to apply it to solving a problem. Employers meanwhile, care what you can do, not just what you know. There’s no doubt that authentic scenarios are a more effective means to develop problem solving and critical thinking skills, but these complex assessments are hard to administer and even harder to evaluate. AI is the disruptive opportunity that may finally be able to complete the circle and provide authentic assessment at scale.

Will Robots Replace Teachers?

One of the biggest arguments against the use of AI in education is that teachers will no longer be required. Advocates for AI disagree. Often, teachers’ capacities are overstretched due to the number of students they need to teach. They also seek ways to impart knowledge with as many of their students as possible, usually employing the traditional lecture-style approach—the least effective way to learn, according to Bloom’s research. But the worst part for teachers is knowing that, inevitably, one size lesson doesn’t fit all, and some students will fall behind.

The undeniable truth is that teachers can not personally grade the work of 500 students on their own. So, for larger classes or online programs, many have already replaced themselves in part by ceding assessment to auto-gradable multiple-choice tests. These tests have been shown to be poor measures of anything beyond factual recall. What Authess aims to do with AI and analytics is bring back performance-based assessments—the results of which are more meaningful and more representative of competency and mastery. AI doesn’t replace teachers, it augments them and provides them a powerful and effective tool to support assessment and feedback to enhance learning.

Shimon Shocken, founding dean of the Efi Arazi School of Computer Science at IDC Herzliya and advocate of MOOCs and education technology, believes “Teachers should be empowered, not replaced”. In fact, those who see the benefits of AI, authentic assessments, and education technology, argue that technological advancements in the classroom will in fact, allow teachers to excel at teaching. How?

For starters, AI solutions can provide educators with real-life information on who is performing above or below standards, allowing them to focus more time on coaching students who need it most, on the specific areas where they need it most. Thus, supporting Bloom’s point on 1:1 learning as being the most effective. Teacher interactions with students become much more relevant and productive to those students. One of the great potential benefits of AI and education technology is that they make it possible for teachers to spend more time with students on a meaningful, individual level, as the time grading coursework and lecturing is greatly reduced. If AI replaces anything, it’s the arduous and tedious act of grading. In its place, AI can provide teachers with higher level reports that focus on where the student missed the mark and where the opportunity for student development is most needed.

Identifying common problem areas for students can be accomplished efficiently by machines, but guiding how that weak area can be improved, of course, requires a human element. That’s when the teacher gets the opportunity to formulate a new approach to teaching that particular pain point.

Leveraging technology in this way also allows teachers to “flip the classroom”: students learn their lesson independently and then come together, with the teacher’s help and supervision, to apply that knowledge while interacting with each other. Individually learning lessons and then working collaboratively in a group is an effective way for students to apply the course content, and it also stimulates the development of power skills such as teamwork, critical thinking, communication, and problem-solving. All of these skills are commonly cited by employers as lacking in recent graduates. There is no age limit for this methodology either: it can just as easily be applied in a K-12 environment, all the way to adult learning.

AI and Education

Artificial Intelligence is already here, and whether we like it or not, students are living in a very different world now. Children born today never may never even drive a car.  The children of this year’s college freshmen will have a very different education experience to their parents and almost unrecognizable from their grandparents, but will share many of the same timeless values. AI-based solutions for education have the capacity to give these future minds the exact level of stimulation they need; the tailored attention from teachers they deserve; and the skills to succeed in their careers.

Why New Hires Don’t Stick

Systemic challenges in human resources and hiring processes are a key explanation for why employee turnover is so high. What can be done to address these issues?

Finding the right candidate to hire is a tough enough challenge, but having that employee flame out, is even more frustrating -- and costly.  The rate this happens though, is depressing -- the probability of a new employee failing after 18 months is close to 50 percent. The financial cost to companies should be an eye-opener, but other factors such as continued productivity, company reputation, camaraderie, and employee morale should not be neglected either.

Systemic Challenges

The reasons why new employees don’t always stay in their jobs vary greatly. Some are role-specific (working for a poor manager or accepting a lower salary than really desired) and must be addressed on a micro level. However, there are certain trends which are systemic to the recruiting and human resources sector, and which can’t be ignored by companies who aim to grow their business.

For starters, only recently have studies been conducted to assess the real cost of high employee turnover. Senior management is (naturally) only going to enforce changes in an inefficient process if it perceives a significant financial loss to the company. Because most companies don’t realize just how much they’re losing from this churn, they don’t feel the need for change.

In large part, recruitment is terribly inefficient because it lacks scientific or data-based assessments — companies rely too heavily on intuition and unstructured interviews, which are often flawed. A great comparison was made in The Guardian newspaper on how scouting for sports relies heavily on statistics and recommendations but how scouting for employees does not. In fact, studies have shown that interviews can do more harm than good.  Judging a candidate’s capabilities largely on interviews instead of on quantifiable data (academic merit, references, etc.) is more likely to lead to poor candidate selection.

Another systemic challenge is that hiring failures often aren’t tracked or quantified internally, which means that it’s impossible to know – and therefore address – the root cause of these failures. Learning where improvements can be made will lead to better candidates being hired, who are likelier to stay with the company longer.

Hiring processes (from applications process to interviews) and employers’ focus on hard skills (knowledge on a particular subject necessary to get the job done, such as proficiency in a particular software program) are unable to evaluate power skills or soft skills. It makes sense to guarantee that qualified candidates possess those hard skills. But in reality, the biggest complaint employers have — namely in regards to recent graduates — is that they are deficient in power skills such as critical thinking, communication, and teamwork.  These foundational skills are integral to success in any role, particularly in today’s workplace.

How to get new hires to stick

What if there were a way to make hiring more data-driven, and therefore more likely to select the most suitable candidates? Authess uses authentic assessment software to evaluate how candidates perform in real-life scenarios compared to benchmarks set by current employees in that company and role. These authentic assessments have two-fold value. First, they represent a clear indicator of a candidate’s approach to actual on-the-job scenarios, as compared to their potential future peers.  Second, these types of tests have shown to reveal more intangible skills like empathy, and power skills such as critical thinking, which are crucial to any role but which only become evident during this type of real-life scenario. Ensuring candidates have the right hard skills and power skills is one challenge these assessments overcome. Meanwhile traditional personality tests, in use by many talent acquisition professionals, have been shown to be poor predictors of job performance.  

Now, imagine a candidate has a degree in a relevant subject for a role, they interview well, and get the job. But without an understanding of their actual skills, there’s no way to know for certain whether they understand how to apply their knowledge in real-life. This is incredibly valuable knowledge for employers to have prior to making an offer. Undergoing an authentic assessment as part of the hiring process also exposes to candidates the type of functions they would be expected to perform, which means they’re less likely to have a false understanding of the job (another cause for leaving a new role).

If hiring processes--both filling the hiring pipeline with qualified candidates and narrowing down to the best candidates--were more scientific and data-driven, there would be a great deal more accountability (and trackability) from a cost perspective for the company. Additionally, because employers are provided with full reports on performance, they can identify positive or negative trends to address internally, if appropriate.

Looking ahead

Authentic assessments are currently being utilized by an array of sectors, due to the positive results they produce — especially in revealing skills that are nearly impossible to identify during the application process. As companies start to appreciate the value of recruiting the right candidates the first time around, more of them will start looking to data-backed tools like Authess to evaluate and redesign their human resources processes. Doing so will increase company profitability, employee satisfaction, and employee retention.