Married to My Work

The phrase "married to my work" is supposed to carry a negative connotation.  It means something like: one's work comes first and foremost in priority, consumes most of one's time (even supposedly free time), that one can never get away from work, it distracts or takes away from family and friend time, and so forth.  The above would be more accurately described as being a "slave to my work," and is obviously not a healthy situation.  On the contrary, an appropriate level of being "married to my work" is a good thing.  But to understand this, we must first reflect on our understanding of marriage itself.

Dating vs. Marriage

What typically is the purpose of dating?  To paraphrase Tom Hanks in Sleepless in Seattle, it is to "try others on" to see how they fit with you.  Do you like the same things?  Do you enjoy talking and spending time together?  Do you have shared values and dreams?  Can you support each other?  It is very much self-focused: what can I or am I getting out of this?  This need not be selfish or self-centered, as it is important if one's ultimate aim is to find a partner for marriage that one finds a good match.  But there is a tendency to focus on having fun, and if things get rough, it might be easier to bail on the relationship instead using the opportunity to grow.

Marriage is different.  Read any marriage book or talk to any marriage counselor.  Once you say "I do", if your marriage is to be successful, you must now begin to focus on the other.  How can you support your spouse's dreams, goals, and pursuits, especially if they are different from your own? The idea behind marriage is that you achieve more and become more having committed to one person.  For better or worse, in sickness and in health, this person is supposed to help you fully realize your potential (and you are supposed to do the same).  You trade short-term, low-commitment advantages for a long-term investment that you can only get from sticking it out with one person for a lifetime.  You get to go deep with someone, knowing that they have committed to you and you have committed to them, and so you can be vulnerable and share yourself with them.

Traditionally, marriage has been about the mutual support of the spouses and the procreation of children.  It is best for you in the long term, and it is fruitful as children are brought into the world and reared.  It is supposed to provide a stable and loving environment for all members to live and grow in.  Yes, it is hard, yes, there are bad times, but in theory, when one looks back on one's life, one will be able to see that one has grown, matured, and gained far more than one has lost in being married as opposed to maintaining a single or frequent dating lifestyle.  The commitment is what allows one, indeed, forces one, to grow and be fruitful, as this is often born out of the challenges of marriage.

No, I am not saying that single people cannot grow and mature without being married or that they will have unfulfilling lives.  Any relationship that is a sustained commitment, an investment, through thick and thin, for the long haul, will encourage and require us to grow, but will also provide life-long enjoyment.  Think of your best friends, your siblings, your parents even.  These are our meaningful relationships.  These are the kinds of relationships that prove to be the most fulfilling.  These relationships are what we are made for.

Application to Work

With the above in mind, what is our approach to our work?  Now I am not advocating placing work above family or other legitimately higher priorities.  But what really is our attitude towards and commitment to work?  Are we dating our work, or married to it (in a healthy way)?  Work is important, and it can be very satisfying, dignifying, and developing in our personal growth and flourishing as a human being.

I have been consulting for the past six years.  The life of a consultant (in my experience, granted, your experience may be different), is that one goes from project to project, company to company, 3, 6, 12 months at a time.  You get in, get what you want out of it (or what your client wants out of it), and then you move on.  It is much like dating.  Either you break up with your client or your client breaks up with you after the work has been accomplished, the relationship has soured, or something else better comes along.  You always have one foot out the door.

Consequently, one's attitude towards work shifts to "what can I get out of this?" for every new project: will it advance my career, will I learn new skills, will I broaden my experience?  The client is asking the same thing: will this consultant advance my work, my project, and my career?  Now some of this is fine and good and healthy, as we do need to think about what is best for us (in a healthy way).  But it is interesting to notice how this can prioritize short term gain over the long term benefits of really committing to a role for the long haul, and encourages breadth of experience at the cost of depth of experience and knowledge.

I recently had the opportunity to go full time and leave consulting behind.  In thinking about whether to accept that opportunity, the analogy to dating and marriage came to mind.  Going full time is much more like marriage.  I would be trading a broad range of experiences for the depth of a single experience.  Instead of having relatively shallow knowledge of a wide variety subject areas, I would become an expert in a relatively narrow field.  Instead of hopping from project to project after my work has been completed, I would get to enjoy the "fruits of my labor" along with the challenges of building and maintaining a long term solution.  I would be committed, knowing that I cannot just walk away if the work gets challenging or coworkers get unpleasant.  I'd be in it, for better, and for worse.  And the role would definitely require more of me both in time and effort.  But hopefully, I'd get to reap the rewards of that long term commitment: greater satisfaction, deep knowledge, something I can point to as having accomplished, deeper friendships with coworkers, and personal and professional growth.

Conclusion

Yes, there is a time for dating your work.  And there are outside work commitments and other considerations that may make a lesser commitment to your work the right decision.  Yes, you should prioritize your family and perhaps other commitments over work. Yes, you shouldn't be overly committed to your work in an unhealthy environment, and you should maintain appropriate boundaries.  Yes, yes, yes, all of that needs to be considered. 

But after considering all of the caveats and addendums and factors in your life, honestly reflect: are you dating your work?  Why?  Is there a good reason?  Are you satisfied with your work?  If not, maybe its because you have never really committed and invested in your work in the ways that are necessary for satisfaction, fruitfulness, growth, and fulfillment in the long term.

If you want to be happy in your work, maybe it's time to get married.

The Interview Games: 10 Tips for Being Successful

Over my relatively short career, I have done many interviews.  Some have been for new jobs while others have been for new clients while working at the same consulting company. While I am sure I still have much to learn about the interview process, here are some insights I have already gathered in my experience in the BI and data analytics job market.

------------------------------------------------------------------------------------------------------------

1. The interview process experience depends greatly on the company culture and role.

In some companies, a culture fit appears to be most important: are you likeable, do you have good "soft" skills.  In consulting, while the broad technical skills are important, the nature of the work requires a lot of personal interaction, self-motivation, discipline, and communication as one works across multiple groups on a regular basis and is only on a project for a short time.  This is what matters most to the consulting company in recruiting employees.  In a more technical full time employment role, the "soft" skills are also important, but the technical skills appear to be much more valued as one works mainly with the same group of individuals in a deeply technical role for a long period of time.

Consequently, the interview style tends to match the nature of the work.  For a consulting position, one looks mainly for personal soft skills.  However, as a consultant interviewing for a technical client position, make sure that you have your technical skills down solid, as these may be the most important skills you have to the client.  You have to have both the soft and hard skills, but mostly it will be your technical skills that are on trial in an interview.  For a technical FTE, one looks mainly for technical skills, but the soft skills are important too, so don't minimize these.

In short, know what kind of company culture and role you are interviewing for, and present yourself accordingly.

2. In person meetings are important for getting an interview.

When possible, it is important to meet with a hiring manager in person before any interview.  This brings to life an otherwise unknown person defined by a piece of paper.  One can better empathize and relate with someone face to face.  And it distinguishes you from everyone else who applied for the position, but never met with the hiring manager.  It is too easy to ignore an email or glaze over a resume.  Don't let that happen to you.  Get a meeting, an "informational", to make that connection.

This is true for both full time positions and for consulting type work.  In consulting, getting in front of a potential client makes all of the difference.  When work becomes available, they will think of you for the position to do that work.

3. A personal connection still matters.

A common respected connection to introduce or recommend you for a position goes a long way in getting that interview, and can even smooth over any difficulties experienced in the interview.  They act as a character and work reference informally, and can root for you and help you prepare.  The fact is, a better candidate may be passed over for another candidate with a personal connection batting for him or her.  Don't ignore this importance, and make the most use of your connections.

In consulting, this is especially true.  Fellow consultants or employees that are known to the client and that can introduce you to the client will help you get the project.  Use the success of your connections to achieve your own success in meeting with clients.

4. Typically, but not always, interviewing is less about assessing ones technical skills and more about assessing one's ability to think and passion for the job.

For technical FTE positions, one may be asked very few technical questions, even though the position is technical.  Why?  Well, most technical questions about coding can be found in a minute through an internet search.  And most technologies can be learned fairly quickly with dedicated study.  Hence, the most important skill for long term success in a role is the ability to think well about a problem and to find an effective solution quickly.  This skill is not so easily learned and is much more valuable nowadays. And if one is not excited about the role, chances are one isn't going to do very good work or be motivated to give one's best.  So conveying passion is necessary.

I say this with the caveat that I have been in interviews where the technical was all that mattered.  My ability or inability to rattle off esoteric code syntax was what determined whether I got the job or not.  This is especially true in consulting.  While your employer may desire you to be a well-rounded individual, a technical client just wants you to deliver using a specific set of skills, and you may be completely judged on how well you can articulate those skills.  Make sure you can.

5. Personality matters, and if you have a personality mismatch with your interviewer, tough luck.  But maybe that is a good thing.

My worst interviews in my experience came as a result of ineffective communication and personality disagreements.  It's hard to interview well when the interviewer is cold, combative, and unclear, but you have to remain warm, excited, professional, and clear.  Perhaps this is even part of the interview, a test to see how well you do under stress and in dealing with a difficult "customer".  Reflecting back now, however, perhaps it is best when those jobs don't work out.  Is it really in my long term interest to work for a team, whether as an FTE or consultant, which has a culture that is negative or in which I just don't fit?  Probably not.

A good job can be characterized by a good project (work/subject matter), good pay (compensation), and good people (coworkers, clients, customers).  Even with good work and good pay, difficult bosses and coworkers can make work miserable.  So don't despair if you and the interviewer don't click.  This may be a blessing in disguise.

6. You have to sell yourself.

Prepare to be a sales person, and the product you are pushing is you.  You can take this in two ways: become everything to everyone you are interviewing with, or put your best foot forward.  I recommend the second route.  If you opt for the first route, you will feel like you are selling your soul in some sense by pretending to be what you are not, and likely, this job will not be a good fit for you anyway.  And people can see through the phoniness that you try to pass off as genuine, so it will likely backfire.  So focus on your strengths, be honest in your weaknesses, and look for jobs or clients that fit what you are excited about and what you can do best.

That being said, you really do need to sell the real you.  You'll probably feel like you are overdoing it, but that's ok.  If you are excited about something, be visibly excited!  Turn gaps in your resume into opportunities for learning.  Explain in detail what you do and what you know.  Be positive and confident.  You need to be likeable.

In short, put the best spin on who you are and what you do in your presentation to the hiring manager or client.  Be true to yourself, but show the best version of yourself that you can.

7. Be prepared for a marathon.

The job search and interview process is grueling.  Be prepared for a long slog and (unless you don't have a current job), wait to begin the process until you can be prepared to put in the effort.  It will feel like working two jobs at the same time.  You need to have the time and energy to do good job searches, prepare for interviews, and conduct those interviews.  If you don't have a good month or two or three to do this in, wait for a better time.  You don't want to hurt your future chances by doing interviews prematurely that do not go well, but which are part of your interview record nevertheless.

As a consultant, one interviews pretty regularly with clients.  But maybe 1 out of 5 of those turns into something.  And with consulting, the stakes are much lower for a bad hire, so the interviews tend to be less intense and grueling.  For an FTE position, maybe 1 in 10 will result in an offer, or perhaps less.  It's a numbers game, and you may have to keep playing for a while before you win in this game of roulette.

8. You can't tell your current boss, until you have an offer.

One can't feel good about the necessary deception (or at least omission) about your job search and interviewing with your current employer.  But what option do you have?  If your job search is known, you may be let go, put on a terrible project, lose a promotion or bonus, etc.  This could especially come back to bite you if you are not successful in finding another job.  So you can't talk about it.  But you still have to go on working as though you will continue to be there long term.  I don't like advocating this duplicity, but I am not sure that there is any other choice here.  In a political working world, one has to be political sometimes.  If you have any better suggestions, I'd love to know them.

Be careful who you trust with knowledge of your job search.  Perhaps you have been blessed with a great manager who cares more about your happiness and long term success than whether you remain a part of the team.  But if that is the case, it is hard to imagine you leaving that situation under normal circumstances.  Best to play it safe if you aren't sure.

That being said, once you have an offer, talk to your boss or manager about it.  You can use the offer as leverage for something more.  "Something more" need not be more money, but it can be whatever reason you might have for thinking about leaving (e.g., promotion, experience).  If you think you can have a good discussion about it, talk to your boss about his or her thoughts on the offer and reasons for going or staying.  He or she may convince you to stay.

9. Great isn't good enough.  You have to be the best

In an employer's market, with tens, even hundreds, of people applying for the same job, getting an interview is an accomplishment.  But even if you get that far, you will still be competing against several others.  It doesn't matter if you can easily do the job and if you are a great fit.  If you aren't the best fit, you won't get the job.  You have to be the best.  Sometimes you aren't, and that is hard, because you didn't do anything wrong.  You just got unlucky, beaten by a better candidate even though you truly did your best and couldn't have done anything more.  Pick yourself up and try again.  If you don't give up, someday, you will be the best and you will get the job.

10. The grass isn't always greener

Why are you interviewing in the first place?  Potential employers will ask you, so you better know why.  Is it for better compensation, more employer engagement, better job experience, a promotion, work-life balance?  What will you gain by leaving?  What will you lose?  Make sure you understand what you really want and the prospects for getting these things.  Then before you decide to interview or leave for other positions, consider how you might bring about or participate in the desired changes in your current role.

If you want a raise or promotion, have you asked for one?  If the work-life balance is bothering you, have you talked with your boss about solutions?  If company engagement is lacking, have you suggested ideas for better communication and engagement?  Changing jobs is difficult, and you especially don't want to trade a mediocre, or even good, known, for a bad unknown that you thought would be great but isn't.

Do an honest assessment of your wants, needs, and expectations, and think realistically about whether these will be satisfied somewhere else, or if you can get them in your current role.  You may be surprised to discover that where you are already is in fact the best place to be all things considered.

--------------------------------------------------------------------------------------------------------------------

Hope that helps.  What do you think?  What has been most challenging and surprising in your job search and interview experience?

Philosophy of Analytics, Lesson One: Begin with the End

Introduction

I have a Masters in Philosophy.  More specifically, I have a Masters in Analytic Philosophy, which is the method of philosophy most popular in the United States and the United Kingdom (as opposed to Continental/Existential/Phenomenological Philosophy).  When I began this blog, I titled it "Philosophical Analytics" to emphasize my interest in both philosophy and analytics (i.e., data related analyses and visualization) and my desire to continue to engage in both disciplines.

Recently, after reading the title of my blog, someone asked me what my philosophy of analytics was.  I had to think a while on this as I had not really formulated my impressions on doing analytics into well formed thoughts, principles, or assertions.  But this is a good question, a perfect question, for me.  This is the first of a series of blog posts that are an attempt to rectify that and to put forth my analytics philosophy thus far.

A Philosophy of Analytics

This blog is not about  how to build a predictive model in R, chart in Excel, or pull a query in SQL.  It's aim is more philosophical and methodological.  We could even call it "meta-analytics".  It will address the why or the why not behind the specific how.  Why should we create this dashboard? Why is this predictive model being used?  We are reflecting on the purpose of using various analytical tools and products.

But we can take a step back even further.  Why do analytics at all?  What is the purpose of analytics? How do we do analytics well?  The answers may be obvious to some, but others may have never asked these questions before.  We know that some people like shiny, flashy, and brightly-colored charts, graphs, slides, and dashboards.  They are impressed by regression lines and cool statistics.  But apart from the job security and budgetary victories that may be scored by throwing analytics left and right, what is the real point of analytics?

This blog post (and others like it) will focus on the nature of analytics: who uses it or does it, what it is, when is it appropriate to use, where should it be used, why is it used, and how best to use it.

Lesson One: Begin with the End

Aristotle specified four "causes" to account for or describe any object:

• the material cause: what the object is materially made of.
• the formal cause: how the object's material is arranged or shaped.
• the efficient cause: what brings about, creates, or changes the object.
• the final cause: the object's purpose, it's end.

Our object is analytics.  It is materially made of Excel sheets, Tableau dashboards, SQL queries, big data stores, R regressions, and Azure predictive models.  It's form consists of colors, lines, plots, and tables placed on billboards, websites, and desktops.  It is efficiently caused by developers from backend to frontend and by users that consume and provide feedback. But it's purpose?  What is it all about? In short:

The end, the purpose, the telos of analytics is this: data-driven decision making.

Why do we make charts or build models?  Why do we collect terabytes of data?  Surely not just because it's fun.  We do these things in order to gain insight into our domain of interest, to understand what is going on. Just because?  No!  We do this because we have to make decisions in our domain of interest, and we want to make good decisions. 

A business needs to understand what it's customers want through sales trends.  It needs to know how to allocate resources, where budgets need to be cut or expanded.  It does so by looking at its sales and financial data appropriately summarized and charted.  A politician needs to know where differing demographics stand on multiple issues so that he or she can most favorably present himself or herself to that demographic.  This is done through collections of voting records that have been grouped according to these demographics and along the lines of key issues.

Perhaps you are simply interested in learning more about a subject (as I often am), and perhaps your explorations have no immediate practical application.  Ok, fine.  Then we can describe the purpose of analytics in a more sophisticated way:

The purpose of analytics is to (1) justify or change beliefs with the use of evidence in the form of data  (2) in order to generate true (or truer) beliefs that can then be used to make decisions related to one's goals, and that, (3) because they are more in accord with reality, are more effective in bringing about one's goals.

 Let's break this down.  First, we are gathering data and presenting it to ourselves or to others to change or strengthen our beliefs about a given domain.  We are trying to understand what really is going on in the world, changing our beliefs about it if necessary.  If the data and subsequent analysis is good, then it will accurately represent reality in a useful way.  For example, we collect weather data and build models that predict what the weather will be like tomorrow so that we can have a true belief about the weather tomorrow.

Now why might that be important?  This leads to the second part.  We take our justified beliefs, supported by the evidence, to make decisions with regards to our goals. If I have the goal of having a good time while hiking outside, what I decide to wear outside will have a direct impact on whether I do have a good time if, contrary to my belief that it will be sunny, it in fact rains and I have decided to hike in a t-shirt and shorts.  If my belief were changed by the evidence that it would in fact rain, then I could make the decision to wear a jacket and have a more enjoyable time.

And this leads to the third part, that data supported decisions are more effective in bringing about our goals. We use data to build a model of reality in a specific domain, and then using that model to represent reality, we make a decision that is lived out in reality.  In the case above, I would look at the data related to weather patterns and historical trends along with the predicted outcomes for what the weather will be like for my hike.  Because the data and models suggest that it will rain, I make the decision to wear and rain jacket.  Consequently, when it does rain, I am not soaked and I still have a good time hiking (which I would not have had if I had gotten soaked).

The goal of my hike was to have a good time.  And by using the analytics related to weather to inform my choice of clothing, I was able to have a good time.  The data and models enabled my hike to be successful, that is, effective, in bringing about my goal.  I was empowered to make a decision more in accord with reality precisely because I had evidence to support my belief that it was going to rain.

Putting It Into Practice: What is Your End?

Ok, so the purpose of analytics is data-driven decision making, that is, to justify/change beliefs that lead to decisions that accomplish one's goals.  So what?  Well, thinking about this purpose of analytics can radically change how you approach your analytics projects.  Most importantly, if the point of analytics is to help one to accomplish one's goals, then before any analytics project can be undertaken, this question must be answered: what are the goals?

Have any of you been told to look at the data, build a dashboard, create a model, and then report back with what you find?  That's like being put into a closet filled with junk and told to find something useful.  What is useful?  It all depends on what one's goals or aims are.  Any number of items may be useful, but they will be useful with respect to certain goals (and not others).

In a business, we can always tie it back to money: increasing profit/revenue/sales or decreasing expenses.  But let's be more specific.  In fact, be as specific as possible.  For your organization, how does it make money specifically?  What role does it play in the larger context of the company?  What are the specific goals of your organization?  What decisions need to be made that could use better data to drive those decisions?

Perhaps your organization saves money by improving code efficiency.  Perhaps your team increases sales by reducing transaction times.  Maybe you improve revenue by increasing the conversion rate of sales through the company website.  Whatever it is, think about the specific goals of your team/organization/company, and then plan your analytics solution to capture, measure, and display the progress in meeting those goals.

Suppose that your organization's goal is to improve click-through rates on the website. Once you know that, you know that you need to gather data related to clicks and visits by users, locations, dates.  And you know that you need to calculate the click-through rate and display this over time (perhaps a line graph) to determine if the organization is improving in meeting its goal. You gather the data with sufficient granularity so that locations that are lagging in click-through rates can receive targeted investment.   Other important decisions can be informed by the data that you have gathered.

Without having this specific goal in mind, you would have no idea what data to gather, what measures to calculate, or what visualizations to create.  And your project wouldn't go anywhere or provide any business value to anyone.

Conclusion

You wouldn't begin a journey without a destination, right?  So don't begin an analytics project without a destination, a specific goal, in mind.  Otherwise, you will wander about aimlessly for a time in the sea of data before your project is cancelled.  Instead, figure out what the end, the purpose, of your analytics will be, and then figure out how to concretely get there.

Before you start, begin with the end.

Blackjack: A Paradox

Introduction

A friend of mine asked me to simulate blackjack deals to verify what he believed would be a new winning strategy in blackjack.  If you don't know the rules of blackjack, see here for the Wikipedia page.  The basic idea he had was that, assuming the dealer hits on 16 or below and stays on 17 or above, the face up cards that lead to a dealer bust should be different from what is typically assumed.

The typical assumption is that the next card is always a 10 in value, as 10s are the most common point value (16/52 are worth 10 points).  Assuming this and the dealer rules, we would expect the dealer to bust the most on face up 2s through 6s.  Why?  Because if a dealer gets a 6 face up, we believe him to have a 10 face down, which adds to 16, which means he has to hit.  But getting another 10 gives him 26 which is a bust.  Similarly down to 2 (2->12->22 = bust).

However, while it is true that 10 is the most common point value for a card, the average point value in the deck is about 7 (higher or lower depending on if the Ace is an 11 or 1 in any hand).  So we should expect any hidden cards to average about 7 points.  But if this is true, then the dealer should bust with a different set of face up cards.  For instance, if we assume the average to be 7, then we would expect 2, 8, and 9 to be the bust cards (2->9->16->23; 8->15->22; 9->16->23).  If you choose a different average calculation, you can get a different set of face up bust cards.

So which is it?  As it turns out, the original basic strategy of assuming the next card is a 10 is the correct approach.  But why?  This leads to the following paradox as both approaches seem intuitively correct:

The paradox: If the average value of a card in blackjack is about 7, why is the basic winning strategy to assume that the next card is a 10?

Data Exploration

The dealer busts about 28.6% of the time and stays about 71.4% of the time.  The dealer does in fact bust most frequently on face up 2s-6s.  See the table below (J=11, Q=12, K=13, A=14). This is based on 100,000 simulations.  5s and 6s are highest at about 43% of the time:

Face Up	Busted	Stay	Busted/Total
2	2671	4964	0.350
3	2828	4742	0.374
4	3136	4522	0.410
5	3338	4320	0.436
6	3294	4445	0.426
7	2047	5720	0.264
8	1817	5945	0.234
9	1771	5880	0.231
10	1670	6005	0.218
11	1623	6106	0.210
12	1679	5950	0.220
13	1716	6002	0.222
14	1012	6797	0.130

So the original basic strategy is correct that the bust cards are 2-6.  And there is a significant break between 6 and 7.  But why is this the case?  Let's do some exploring.

 Averages:

The average value of the cards in a dealer's hand when he busts is 7.03.  When he stays, its  7.61.  Breaking this down by face up card, we have the below table:

Face Up	Busted	Stay	Difference
2	6.076	5.548	0.529
3	6.383	5.696	0.688
4	6.726	5.814	0.911
5	7.052	5.877	1.175
6	7.500	6.350	1.150
7	7.162	7.380	-0.217
8	7.283	7.878	-0.596
9	7.467	8.359	-0.892
10	7.579	8.869	-1.290
11	7.597	8.878	-1.281
12	7.570	8.869	-1.299
13	7.586	8.888	-1.302
14	5.419	8.504	-3.085

Interestingly, the bust averages for 2-6 are higher than the stay averages, but the opposite is true for 7-14.  Why is this the case?  Perhaps another table of the average count of cards will help:

Face Up	Busted	Stay	Difference
2	3.946	3.517	0.429
3	3.783	3.422	0.361
4	3.606	3.338	0.269
5	3.478	3.284	0.194
6	3.296	3.047	0.249
7	3.399	2.580	0.819
8	3.317	2.467	0.850
9	3.233	2.366	0.867
10	3.174	2.257	0.917
11	3.169	2.255	0.915
12	3.184	2.256	0.928
13	3.172	2.255	0.917
14	4.454	2.520	1.934

Busts on average require more cards than stays, which makes sense, because a bust goes over 21 while a stay is between 17-21.  On average, it takes more cards to get a higher value, hence, getting over 21 takes more cards than getting between 17 and 21.

However, there is a big break in average counts of cards between face up 2s-6s and face up 7s-14s that matches the break in average values of cards.  Why should this exist?

If we start with the assumption that the next card (or any hidden card) is a 10, then this is to be expected.  With that assumption, a face up 6 or below is a total of 16 or below (hit) while a 7 face up is a total of 17 (stay).  A hit requires one more card at least, so it makes sense that a large break occurs between 6 and 7. 

Also,  the higher the face up card, the more likely a stay is with only 2 cards, meaning that each card is on average larger than a similar stay with 3 cards.  For example, a stay of 17 with a 10 and 7 averages 8.5 per card while a stay of 17 with an 8, 2, and 7 averages 5.7 per card.  As the face up card gets smaller, the amount of cards needed to get to a stay or bust increases, since the cards are worth less, hence, the averages get smaller.

But this isn't the whole story.  The first table really comes from the combination of the second table and another table, the average hand value for the dealer by face up card.  We can see that the busted averages are largely the same with the highest value at 6.  The stay averages are also similar with a notable low value at 7.  The difference between the busted total value and the stay total value shows a peak of 5.6 at a face up value of 7, and this decreases to either side.

This makes sense, since a 7 face up (and a hidden 10) is the lowest possible stay, while a 6 face up (with a hidden 10) is the highest hit.  So we should expect the greatest difference between busts and stays to be right around the 6 and 7 divide.

Face Up	Busted	Stay	Difference
2	23.325	18.917	4.408
3	23.403	18.974	4.429
4	23.570	18.967	4.603
5	23.843	18.920	4.923
6	24.159	18.761	5.398
7	23.740	18.100	5.640
8	23.692	18.466	5.226
9	23.732	18.865	4.867
10	23.765	19.263	4.502
11	23.792	19.276	4.516
12	23.783	19.261	4.522
13	23.784	19.276	4.507
14	23.712	19.528	4.185

So a large difference in the middle combined with decreasing average counts of cards in the dealers hand makes for a switch in the difference of average card values at the 6/7 divide.  And all of this makes sense when we expect the next (or hidden) card to have a value of 10.

Counts of 10s:

Is assuming that the next (or hidden) card is a 10 in value a good assumption? In a way, yes.  71% of dealer hands have a 10 in them.  Thus, it is reasonable to assume that the dealer will have a 10 face down or at least coming as a third card.  That is, most of the time, the dealer will have a 10 in the hand.

Here is a more detailed breakdown:

Count of 10s	Busted	Stay
0	4154	24520
1	15002	37876
2	9446	9002

The dealer busts only 4% of the time without any 10s.  That is, only 15% of busts lack a 10 valued card.  That means that 85% of busts have at least one 10 involved.  With stays, only 34% of them lack a 10. 

If we remove the possibility of getting a 10 valued card as the face up, then 59% of hands will still have a 10 valued card hidden or coming.  In greater detail:

Count of 10s	Busted	Stay
0	4154	24520
1	12587	22815
2	5173	0

81% of these busts have a 10 involved.  Stays are split nearly 50-50 on having a 10 or not.  When a 10 is involved, 43% of the time the dealer busts.

Clearly, one should expect the dealer to get a 10 valued card.

Combining Averages with 10s

Suppose the dealer gets one 10 valued card (not the face up), which happens 51% of the time.  What happens then?

• Let's assume the other card besides the face up card is a 7 (the average).  If the 10 comes first, then a bust comes on face up 5 and 6, which are the most likely to bust.  If the 10 comes second, then 5-9 are busts.
• If the other card is an 8 (the median), a bust happens on 4-6 with the 10 coming first, and 4-8 when it comes second.
• If the other card is a 10 (the mode) (7% of the time), then a bust happens on 2-6.

So we see that busting on 5-6 is very likely in all circumstances, and then depending on what we take to be the best representative of the set (mean vs. median vs. mode), additional face up values lead to busting.

When we combine all of these together, it is not surprising that we do get the following result of bust averages by face up value, now ordered from most likely to least:

Face Up	Busted	Stay	Busted/Total
5	3338	4320	0.436
6	3294	4445	0.426
4	3136	4522	0.410
3	2828	4742	0.374
2	2671	4964	0.350
7	2047	5720	0.264
8	1817	5945	0.234
9	1771	5880	0.231
13	1716	6002	0.222
12	1679	5950	0.220
10	1670	6005	0.218
11	1623	6106	0.210
14	1012	6797	0.130

Still, if we had to pick one number to represent the set of values, it seems that 10 (the mode) is the best number to do so, and not the mean or the median.  This best aligns with the face up value busts.  But why?

Resolving the Paradox: A Statistical Lesson

If assuming that a hidden card is a 10 in value is the right approach, why is assuming that the next card is a 7 in value a bad approach?  In other words, why does the mode take priority over the mean in representing the distribution of values of cards in blackjack?

The answer lies in the distribution of values in blackjack.  A hidden assumption in my friend's approach is that the average value of the cards was from an approximately normal distribution.  If that were the case, then we could expect the next card to be, on average, a 7, because in fact, the average card would be about a 7.

However, blackjack values are not normally distributed.  There are 4 times as many 10s as there are of any other card value.  As a result, using the average value of the cards to represent any hidden card values is misleading.  This is a misuse of the mean to represent the sample.

To make this point more obvious, suppose we played a game in which every card was worth 200 points except for the 2 of clubs and 2 of spades, which were both worth -5000 points.  The average value of the cards in this set is 0 points.  However, most of the time, the next card played will be worth 200 points. And no card is in fact worth 0 points.

In this circumstance, it makes sense to treat every next card as being worth 200 points, knowing full well that 2 of the cards will be worth much less.  Why?  Because in fact, most of the time, the next card will be 200 points.  The best strategy for winning the game will likely assume that every hidden card is worth 200 points.

Conclusion

Going back to blackjack, and given the foregoing, the best simple assumption does seem to be that any hidden card is a 10 in value.  If we make this a bit more complex, the assumption becomes that the dealer will get at least one 10 valued card.  So if this is not the face up card, we should expect it to be the face down card or a hit card.  This, combined with what we have discovered about the mean and median, leads us view face up cards 2-6 as bust cards, with 5s and 6s being especially important.

The statistical lesson to be learned is that one must be careful in using the mean (or median or mode) to represent the dataset.  Do a sanity check.  Look at the data.  Does it make sense, or does it mischaracterize the data?  Using the word "expected" helps.  Does the mean represent what I expect any unknown value to be, or is the mode or something else a better representation of what is expected? 

Context is also important.  Saying that the average value of a data set is X really does not tell us much about that dataset.  Putting that into the context of the distribution, standard deviation, skew, mode, median, and other various statistical measures and descriptions can help us understand what the data is really like, and how we can best use that data to make good decisions.

It has been said that there are three kinds of lies: "lies, damned lies, and statistics."  Make sure your statistics do not misrepresent the true nature of the data and become worse than damned lies.