Accessibility and Algorithms

Recently, I came across this video (https://www.youtube.com/watch?v=lJYufx0bfpw) that explains a concept used to organize very VERY large data sets. It has the incredibly compelling title of A problem so hard even Google relies on Random Chance.

The td;lr is that you can use the probability of how rare the rarest outcome is to estimate the size of a dataset. If you know that the longest string of heads or tails in a marathon coin flipping session was 5 in a row then you probably didn’t flip the coin that many times. But if the longest streak is 20 then you probably flipped it for a very long time. According to the video, this approach typically can get ranges within 2% accuracy.

One of the things I love learning about most are novel ways to solve problems and brain teasers. Computer science appears to have a treasure trove of these types of problems and most importantly, a large number of people are interested in them and explain them on the internet.

No matter how clever your piece of knowledge is, the collective usefulness of that piece of knowledge is a factor of how many people can actually use it. And in that respect, the internet as we know it today is an order of magnitude shift for the collective intelligence of our species. It’s likely that AI tools delivers the next order of magnitude shift as it layers on the accessibility that the internet brings.

I’d be remiss if I didn’t mention the more common approach of “exploit an advantage” based on tools and techniques to gain the upper hand in the pursuit of profit.

This reminds me of a famous exchange that Steve Jobs had with with Andy Grove (Intel) about favours and sharing insights.

Excerpt from the beautifully built out site: https://book.stevejobsarchive.com/

Email Exchange Between Steve, Intel CEO Andy Grove, and an Intel Engineer

“I have changed my position 180 degrees.”

As Pixar became a leader in graphics, Steve and his mentor, Intel CEO Andy Grove, discussed how Intel might learn from Pixar. When an Intel engineer tried to follow up, Steve resisted.

From: [Engineer 1 at Intel]

To: Steve Jobs

Cc: Andy Grove

Subject: Pixar-3D graphics

Date: September 22, 1995, 2:04 p.m.

Steve,

Andy asked me to look into what we should do in dramatically improving the Intel architecture platform’s 3D graphics performance. He indicated that you and key people at Pixar like Ed Catmull have lots of good ideas on what we should do in this area. I actually contacted Ed several months ago but he was real busy and cannot commit to meeting until after Sept. As you know, I am in charge of microprocessors at Intel.

I have located several key Intel 3D experts. One of them, [Engineer 2], came from Sun over a year ago. I would like to have a meeting (at Next, Intel or Pixar) with you, Catmull, and others with me and our graphics experts to discuss your ideas and map out what our action plans are. I am in Tokyo next week but will be back in my office on Monday Oct. 2. I will ask my admin to contact your office to set that meeting up.

Thanks.

[Engineer 1]

From: Steve Jobs

To: [Engineer 1]

Subject: Pixar-3D graphics

Date: September 23, 1995, 7:11 p.m.

[Engineer 1],

Pixar does indeed possess the knowledge to enable Intel’s processors to render 3D graphics at much high performance and quality. These “secrets” could definitely make their way into future Intel general purpose processor hardware.

We believe this single capability is the key for Intel to dramatically enlarge the PCs market share in the consumer market – by significantly surpassing the dedicated gaming machines (Sega, Nintendo, Playstation, etc) graphics capabilities.

Pixar’s secrets were invented through significant investment over ten years or more, and we value them highly. Even without the secrets implemented in the processor, Pixar can gain significant competitive advantage and differentiation through implementing them in software. By disclosing the “correct” way to do high quality, high performance graphics, Pixar will lose much of this to any and all competitors, with no work on their part. Hence, the need for compensation.

What does Intel propose to give Pixar for disclosing and licensing its secrets to Intel?

Steve

From: [Engineer 1]

To: Steve Jobs

Cc: Andy Grove

Subject: Re: Pixar-3D graphics

Date: September 25, 1995, 11:22 a.m.

Steve,

We would very much like to have our meeting, but I will put that on hold based on your input. We talked to many key people on ideas to improve the microprocessor capability with the aim that this will benefit the whole industry, and everyone will benefit. We have not entered into any financial arrangement in exchange for good ideas for our microprocessors in the past and have no intention for the future.

[Engineer 1]

From: Steve Jobs

To: [Engineer 1]

Cc: Andy Grove

Subject: Re: Pixar-3D graphics

Date: September 25, 1995, 5:29 p.m.

This approach has not served you well in the past, as evidenced by your poor graphics architectures and performance. Maybe you should think of changing it for the future…

Steve

From: Steve Jobs

To: Andy Grove

Subject: Re: Pixar-3D graphics

Date: September 25, 1995, 10:27 p.m.

Andy,

Maybe it’s just me, but I find [Engineer 1]’s approach extremely arrogant, given Intel’s (his?) dismal showing in understanding computer graphics architectural issues in the past…

If I were going to make hundreds of millions of something, I sure as hell would be willing to pay for the best advice money could buy… Any[way], this isn’t a sales pitch; I just wanted you to know what I thought, as always.

Best,

Steve

From: Andy Grove

To: Steve Jobs

Subject: Re[2]: Pixar-3D graphics

Date: September 26, 1995, 3:12 p.m.

Steve,

I am firmly on [Engineer 1]’s side on this one. He is taking your offer to help us very seriously, rounded up the best technical people and was ready to go when you introduced a brand new element into the discussion: money.

You and I have talked many times about this subject; you never suggested or hinted at this being a commercial exchange. I took your offer to help us exactly as that: help, not an offer of a commercial relationship.

You may remember, that from time to time I offered suggestions that pertained to your business. Examples range from porting NextStep to the 486 - - which was in our interest too - - to my presentation to your staff on repositioning NextStep beyond that. I am not suggesting that these are comparable in value to your expertise in graphics, but I gave what I had, put some thought into the problem I saw you were facing - - and it never entered my mind to charge for it. In my view, that’s what friendly companies (and friends) do for each other. In the long run, these things balance out.

I am sorry you don’t feel that way. We will be worse off as a result, and so will the industry.

Regards,

a

From: Steve Jobs

To: Andy Grove

Subject: Re[2]: Pixar-3D graphics

Date: October 1, 1995, 3:50 p.m.

Andy,

I have many faults, but one of them is not ingratitude. And, I do agree with you that “In the long run, these things balance out.”

Therefore, I have changed my position 180 degrees - - we will freely help [Engineer 1] make his processors much better for 3D graphics. Please ask [Engineer 1] to call me, and we will arrange for a meeting as soon as the appropriate Pixar technical folks can be freed up from the film.

Thanks for the clearer perspective.

Steve