tag:blogger.com,1999:blog-53289155210890853852024-02-18T22:07:38.054-08:00Cursory KnowledgeExploring how a little knowledge and a lot of questions lead to something profound... or at least interesting.Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.comBlogger86125tag:blogger.com,1999:blog-5328915521089085385.post-50161686493612056342017-05-15T09:40:00.001-07:002017-05-16T22:01:28.534-07:00Healthcare Spending in the US - What is the Problem?<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgCvLP7gci-w4WHo7WdzP4xwVcfVS2BXu1iXNOGg9w3RhBIL2x6osnk5sUFW721kBMubkTaHbmER5sj1mJSN0_bme77Q9EKoPzKCsIR3OwB_I3d8W-AGHxV3HOxuQCu_EM_whGyxhdBHjn5/s1600/fake+headline+-+healthcare.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="thanks to fodey.com for the image" border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgCvLP7gci-w4WHo7WdzP4xwVcfVS2BXu1iXNOGg9w3RhBIL2x6osnk5sUFW721kBMubkTaHbmER5sj1mJSN0_bme77Q9EKoPzKCsIR3OwB_I3d8W-AGHxV3HOxuQCu_EM_whGyxhdBHjn5/s320/fake+headline+-+healthcare.jpg" title="healthcare is expensive" width="260" /></a><span style="font-family: "arial" , "helvetica" , sans-serif;">It's not news that healthcare is expensive. Here is a quick look into <b>what </b>is going on. <b>Why </b>this is going on is a different question that I'll come back to.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I looked through World Bank WDI data (<a href="http://data.worldbank.org/data-catalog/world-development-indicators">link</a>) to visualize three spending areas that get significant attention: </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">- Education expenditures</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">- Healthcare expenditures </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">- Military spending. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">This data is limited to the United States from 1995 - 2014. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">Figures 1 and 2 below show the results.</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Education and military spending, as a percentage of gross national product (GNI), has remained flat for the last 20 years. Education has remained around 5% and military spending has remaining around 3.5% (with a temporary jump in 2009 to almost 5%).</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">However, healthcare spending has risen as a percentage of GNI over this same time by 4% (from around 13% to over 17%).</span><br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtolLUN4tFK2vsjCvCw4o9kefUIwA4RNcKm18i4WqA39TDev2uIiwC-7weidJmBJ5n5PgYq98T3DQ2blqwVGYSWLCUOwztoGZW_uKYElmSTzdjnlbgycgkjpiwwmCNEde0TyEk5Ss4daUq/s1600/USSpending%2528percent%2529.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="251" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtolLUN4tFK2vsjCvCw4o9kefUIwA4RNcKm18i4WqA39TDev2uIiwC-7weidJmBJ5n5PgYq98T3DQ2blqwVGYSWLCUOwztoGZW_uKYElmSTzdjnlbgycgkjpiwwmCNEde0TyEk5Ss4daUq/s640/USSpending%2528percent%2529.PNG" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fig 1 - Expenditures as % of GNI (note: for the US, GNI and GDP are very close so I have used the two interchangeably)</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: "arial" , "helvetica" , sans-serif; margin-left: 1em; margin-right: 1em;"></span></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">When converted to dollars, the increase can be seen more clearly.</span><br />
<ul>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Education increases from ~$364 billion to $904 billion</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Military increases from ~$271 billion to ~$627 billion</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Healthcare increases from ~$995 billion to over $3 trillion</span></li>
</ul>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiW96BvetF3xlZT032qnQ20_csnMHXjbaZ6suNXoSdSBR_MAxnEfe6rQmXP5b6YW8lnkiBjx75k0LMpHqYcbvWhcSRkzKUqOzqil4NKMSC4F9WJQPtbOv2K731Y-z110LH7IKL-F-H6x-3D/s640/USSpending%2528dollars%2529.PNG" style="margin-left: auto; margin-right: auto;" width="640" /></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fig 2- Expenditures in US$</td></tr>
</tbody></table>
<span style="clear: left; font-family: "arial" , "helvetica" , sans-serif; margin-bottom: 1em; margin-left: 1em; margin-right: 1em;"><span style="color: black;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiW96BvetF3xlZT032qnQ20_csnMHXjbaZ6suNXoSdSBR_MAxnEfe6rQmXP5b6YW8lnkiBjx75k0LMpHqYcbvWhcSRkzKUqOzqil4NKMSC4F9WJQPtbOv2K731Y-z110LH7IKL-F-H6x-3D/s1600/USSpending%2528dollars%2529.PNG" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: 1em;"><br /></a></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">The Centers for Medicare and Medicaid Services maintains a breakdown of healthcare spending (</span><a href="https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/NationalHealthExpendData/NationalHealthAccountsHistorical.html" style="font-family: arial, helvetica, sans-serif;">link</a><span style="font-family: "arial" , "helvetica" , sans-serif;">). The summary of spending (</span><a href="https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/NationalHealthExpendData/Downloads/highlights.pdf" style="font-family: arial, helvetica, sans-serif;">PDF</a><span style="font-family: "arial" , "helvetica" , sans-serif;">) shows that over 50% goes towards hospital and physician services.</span><br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><img border="0" height="392" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgCsmI_VfSsheIUYHfKF-kW_a9jtfiUXBvaNAJzVY9KWDuVIUWIh4cHLLyX8WPPNE6cJgxpfI8Qrph3kTjc5qXoKgP11FoJ-XT4EFL2cWLjU2iOcvDfiPmHm_CQqg05TDA10qjuZ6xf-8qE/s640/Breakdwon+of+top+healthcare+spending+categories.PNG" style="margin-left: auto; margin-right: auto;" width="640" /></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fig 3 - Breakdown of healthcare spending in the US in 2015</td></tr>
</tbody></table>
<span style="font-family: "arial" , "helvetica" , sans-serif;">The CMS summary report states that:</span><br />
<ul>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Hospital care spending increased by 5.6% in 2015 while prices only increased 0.9%. This means hospital spending was driven by increased usage and intensity of services.</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Physician services increased by 6.3% in 2015 while prices declined by 1.1%. This means that physician spending was driven by increased demand.</span></li>
</ul>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Taken together this suggests that <b>most of the expenditures are occurring in areas that are being driven up by people needing more or more intensive (and expensive) care</b>. </span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">The interesting next question is: what is driving that increased need or intensity of care and how can those root causes be addressed?</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Bonus stuff: A Tableau Public visualization of this data (<a href="https://public.tableau.com/views/USSpendingTrends/Story1?:embed=y&:display_count=yes">here</a>).</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<br /></div>
<div class="tableauPlaceholder" id="viz1494995799070" style="position: relative;">
<noscript><a href='#'><img alt='Healthcare Spending in the US ' src='https://public.tableau.com/static/images/US/USSpendingTrends/Story1/1_rss.png' style='border: none' /></a></noscript><object class="tableauViz" style="display: none;"><param name='host_url' value='https%3A%2F%2Fpublic.tableau.com%2F' /> <param name='site_root' value='' /><param name='name' value='USSpendingTrends/Story1' /><param name='tabs' value='no' /><param name='toolbar' value='yes' /><param name='static_image' value='https://public.tableau.com/static/images/US/USSpendingTrends/Story1/1.png' /> <param name='animate_transition' value='yes' /><param name='display_static_image' value='yes' /><param name='display_spinner' value='yes' /><param name='display_overlay' value='yes' /><param name='display_count' value='yes' /></object></div>
<script type="text/javascript"> var divElement = document.getElementById('viz1494995799070'); var vizElement = divElement.getElementsByTagName('object')[0]; vizElement.style.width='1020px';vizElement.style.height='1033px'; var scriptElement = document.createElement('script'); scriptElement.src = 'https://public.tableau.com/javascripts/api/viz_v1.js'; vizElement.parentNode.insertBefore(scriptElement, vizElement); </script>
Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-30828591431094019782017-05-09T17:17:00.000-07:002017-05-09T17:17:12.263-07:00The Essence of Product Life Cycle (PLC)<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjARRjQ6CV1lNp6BrBJPyK5FwIEESxvJzJqEZhjZJztDJUzEluoEp1PayZkxIOxbcgSAZ1c-HomCRX3MMRdrdM5P1Hzy2Wt99WuHu4FDoabSituIQnfgIOpvXRFg9GpJqI-pXlp9BM-1f8L/s1600/PLCSummaryTable.PNG" imageanchor="1" style="clear: right; display: inline !important; margin-bottom: 1em; margin-left: auto; margin-right: auto; text-align: center;"><img border="0" height="227" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjARRjQ6CV1lNp6BrBJPyK5FwIEESxvJzJqEZhjZJztDJUzEluoEp1PayZkxIOxbcgSAZ1c-HomCRX3MMRdrdM5P1Hzy2Wt99WuHu4FDoabSituIQnfgIOpvXRFg9GpJqI-pXlp9BM-1f8L/s400/PLCSummaryTable.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fig 1 - PLC Cheat Sheet</td></tr>
</tbody></table>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><u>The summary</u></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Here is a cheat-sheet for the major steps in a product life cycle (Fig 1). It covers four ideas:</span><br />
<br />
<ol>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">What are the phases in a life cycle?</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">What is the top level goal of each phase?</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Who are the key actors?</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">What are the actors trying to do in each phase?</span></li>
</ol>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><u><br /></u></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><u><br /></u></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><u>But what about agile?</u></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Agile is a methodology for answering </span><b style="font-family: Arial, Helvetica, sans-serif;">some </b><span style="font-family: "arial" , "helvetica" , sans-serif;">of the questions in the life cycle. Agile is not a substitute for a proper life cycle process (more on this in a minute in "cycles repeat"). Choosing whether to follow an agile method or a more traditional waterfall method depends, I have come to believe, on the cost of developing requirements vs the cost of validating those requirements (Fig 2).</span><br />
<br />
<div style="text-align: right;">
</div>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHqIZvJcAxAUidKh8JJ8XLaFsoJjqtuGu1cZTBJVVIFE6cxejZXxvhHnGUnjJnQVUKqgyiWu3L6XlnelwIE7d_9GpfLT8ZY0Ure1oy88RGiDZEWuWmFJuyCnGzmdL3Nmav3mSfmKLt6MLF/s1600/Agile+in+PLC.PNG" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="227" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHqIZvJcAxAUidKh8JJ8XLaFsoJjqtuGu1cZTBJVVIFE6cxejZXxvhHnGUnjJnQVUKqgyiWu3L6XlnelwIE7d_9GpfLT8ZY0Ure1oy88RGiDZEWuWmFJuyCnGzmdL3Nmav3mSfmKLt6MLF/s400/Agile+in+PLC.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fig 2 - Agile vs Waterfall Development depends on cost.</td></tr>
</tbody></table>
<ul>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">If the cost of developing your product to a point where the requirements can be tested is <b>LOW</b>, then it pays to adopt an agile approach. <i><span style="color: blue;">Optimize for speed to market because each iteration is cheap</span>.</i></span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">If the cost of development or testing is <b>HIGH</b>, then it pays to invest more time getting the requirements right before paying to develop and test them. <i><span style="color: blue;">Optimize for learning per unit cost because each iteration is expensive</span>.</i></span></li>
</ul>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">The specifics of how much is required to get a testable concept vary from case to case. </span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">For example: The cost of developing and testing a deep UV optical system to determine if it can collect enough data for the detection algorithms to flag a sub-wavelength sized pattern difference is quite high. The cost of testing different parameter entry field orders to determine which one causes more users to follow the correct setup procedure is relatively low. Hence semiconductor capital equipment hardware is not developed according to agile methods while the software that runs the hardware can be developed in an agile way.</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i>[More on this in an <a href="http://cursoryknowledge.blogspot.com/2012/08/cant-test-this-ab-testing.html">earlier post</a>.]</i></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><u>Cycles repeat</u></span></div>
<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXxeu5C5w_LcT6RrHQ1dXjv5iqShafuR5lym6VeMEWEE_1_lX9dAcRbAgOwgVWsjse4z4l9VzS_htwbfSHjM5oJUscC6_HBLc3h7VW8q0muXBfChXhvJAq6IlXnGkXtyWyva3OLSVgy3hF/s1600/PLC+Loops.PNG" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto; text-align: center;"><img border="0" height="226" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXxeu5C5w_LcT6RrHQ1dXjv5iqShafuR5lym6VeMEWEE_1_lX9dAcRbAgOwgVWsjse4z4l9VzS_htwbfSHjM5oJUscC6_HBLc3h7VW8q0muXBfChXhvJAq6IlXnGkXtyWyva3OLSVgy3hF/s400/PLC+Loops.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fig 3 - Iterations in a Product Life Cycle</td></tr>
</tbody></table>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Learning at each phase will determine whether to proceed to the next phase or to return to a previous phase (fig 3).</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I think the most interesting part is that there are basically only two questions here:</span><br />
<br />
<ol>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Am I solving the right problem?</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Is there a better approach to solving the problem at hand?</span></li>
</ol>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Under the right conditions, agile is good for moving quickly through the iterations required to answer these questions. However, agile methods, in themselves, won't guide you to ask the right questions at the right time - that is what a product life cycle is for.</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Extra stuff:</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">The slides that these images come from are embedded below.</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<iframe allowfullscreen="true" frameborder="0" height="569" mozallowfullscreen="true" src="https://docs.google.com/presentation/d/1ie1GCLKCN6W-51rmexbpK1TzyP--dx0nQLRykp8N54g/embed?start=false&loop=false&delayms=3000" webkitallowfullscreen="true" width="960"></iframe>Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-72885314076024431192017-05-04T14:19:00.001-07:002017-05-04T14:19:41.749-07:00Rogue AI and Human Ego<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZXIX_X9zeZkk2O3Fvle7bXj1qica3YPDlrjR-youQ7s5ZjU5preneJ8azuaGPKNSwkzJTH45AB7aZlos45HbEi7NxC-GdpcW-s2TOFhtsktHVHX227MrL9Qm55SCTsNYrR8Xk-ZcHgZ_P/s1600/shodan.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZXIX_X9zeZkk2O3Fvle7bXj1qica3YPDlrjR-youQ7s5ZjU5preneJ8azuaGPKNSwkzJTH45AB7aZlos45HbEi7NxC-GdpcW-s2TOFhtsktHVHX227MrL9Qm55SCTsNYrR8Xk-ZcHgZ_P/s400/shodan.jpg" width="400" /></a></div>
<span style="font-family: Arial, Helvetica, sans-serif;">In a bout of good conversation with a friend, we ended up asking the question:</span><br />
<blockquote class="tr_bq">
<span style="font-family: Arial, Helvetica, sans-serif;"><i>How do you hold an artificial intelligence (AI) accountable for its actions?</i></span></blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;"> "Punishment!" We said; but...</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">How does one punish an AI?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">The same way one would punish a person: Take away something that it cares about.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">What does an AI care about such that taking it away will cause a change in behavior?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">Why would taking something away cause a change?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">What would even motivate an AI in the first place?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">"hmmm...." We said...</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">What if an AI's motivation worked in a completely different way from a human's motivation?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">What if the AI's value system was built like an insect hive's? Where no member could even <b>conceive </b>of the idea of performing a "bad" (i.e. independent, self-serving, coming at the cost of another) action?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">Does an ant colony ever have a rogue ant problem?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">(I think it safe to say that humans have rogue human problems, even without AI.)</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Perhaps the rogue AI problem comes from the hubristic assumption that a "good" (i.e. functional, effective, general) AI, needs to be modeled on human intelligence? </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Perhaps, just as a fish doesn't know water, we are blind to our primate sense of fairness and justice, evolved to manage exactly the kind of intelligence we happen to have. Because of this, we can't see an alternative to the idea that</span><span style="font-family: Arial, Helvetica, sans-serif;"> a human based intelligence must come with a human based motivational system, including individuality and rule questioning behaviors.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Are we, in fact, creating the control problem by assuming that the intelligence we create should function like our own?</b></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">(Kevin Kelly has something to say about this from a slightly different angle: <a href="http://kk.org/thetechnium/ai-or-alien-intelligence/">AI or Alien Intelligence</a>)</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com1tag:blogger.com,1999:blog-5328915521089085385.post-38429482694589582172016-05-22T10:54:00.002-07:002016-05-22T10:54:42.010-07:00Why we Can't Kill Email - The Essence of Knowledge Work<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSOUO6LDXXSc4kNUAoqyPmDpmP0PzpiPlo3Ffu7T_wlADED1N9vAXUpvfGMOBALOudmIDMorcVDCH7a4kt_VHS7VTzmmnGeMfmQdpcBzcbp-NEsY-hdlmGZFpyyzkHEOTdobLkaxbXGGbf/s1600/Email+is+a+buffer+diagram.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="Knowledge vs manufacturing workflow" border="0" height="270" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSOUO6LDXXSc4kNUAoqyPmDpmP0PzpiPlo3Ffu7T_wlADED1N9vAXUpvfGMOBALOudmIDMorcVDCH7a4kt_VHS7VTzmmnGeMfmQdpcBzcbp-NEsY-hdlmGZFpyyzkHEOTdobLkaxbXGGbf/s400/Email+is+a+buffer+diagram.JPG" title="Knowledge vs manufacturing workflow" width="400" /></a>I was thinking about <u>Getting Things Done</u> by David Allen and a course I had taken about Outlook best practices. The main idea behind both is this - Get it out of your brain by applying "The 4 Ds:"<br />
<br />
<ol>
<li>Do it now (in less than 2 minutes)</li>
<li>Delegate it to someone else</li>
<li>Defer it (to a date)</li>
<li>Delete it</li>
</ol>
<div>
I have been tweaking how I implement this process over the years and have settled on a process which works for me:</div>
<div>
<ul>
<li>Use email as a buffer to manage messages. </li>
<ul>
<li>Information from others usually comes in this way. I'll also email myself things that need to be dealt with as they come up.</li>
</ul>
<li>Deal with the content of the email immediately, delete it or create / edit a task related to the content of the email. </li>
<ul>
<li>I will usually break down each task into multiple steps required to achieve the desired outcome to the degree possible at the time.</li>
</ul>
<li>Use the task list as a work queue. </li>
<ul>
<li>I review the list regularly to understand everything that needs to be done and by when. </li>
<li>I amend the details of each task to use as a record of communications and actions.</li>
</ul>
<li>Put working blocks into my calendar based on the tasks.</li>
<ul>
<li>Depending on the task scope and timeline, this means blocking out many chunks across many days. </li>
<li>It ensures that I strike the right balance between standing meetings, appointments and the "real" work... If the balance is off, the calendar will bring that fact forward quickly.</li>
</ul>
</ul>
<div>
This flow, I realized, is parallel to the flow used to manufacture goods in a factory:</div>
</div>
<div>
<ul>
<li>Incoming materials are <b>buffered </b>in trucks and at the dock.</li>
<li>The materials are <b>queued </b>at the appropriate stations in the factory according to what they are used to manufacture.</li>
<li>Processing is <b>scheduled </b>and performed.</li>
<li>The end result is shipped out (or re-queued for a subsequent process).</li>
</ul>
<div>
You don't have a manufacturing line that can't do these things: Buffer, queue, schedule.</div>
</div>
<div>
<br /></div>
<div>
Knowledge work is similar: communications are queued into tasks which then must be scheduled and performed.</div>
<div>
The problem is that the overhead of working this way is high. It's "easier" to just remember what you heard, what you have to do and then do it.</div>
<div>
Up to a point...</div>
<div>
When there is too much to do this system breaks.</div>
<div>
<br /></div>
<div>
Email helps because it seamlessly buffers the communications and CAN be used as a task list. Email won't die because a reasonably low friction replacement that does the end-to-end job doesn't exist.</div>
<div>
Chat clients don't replace email because email, as it is used at work, isn't about communication. It's about the entire Knowledge workflow.</div>
<div>
<br /></div>
<div>
<b>Make the whole workflow better and you'll kill email (as it is used today).</b></div>
<div>
Though, I suspect, if you make the workflow sufficiently good, you'll find that email is a friend, not an enemy.</div>
<div>
<br /></div>
Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-45915520814382100572014-03-30T16:16:00.000-07:002014-03-30T16:16:52.616-07:00What are you really doing when you're doing what you're doing?I was re-reading <u><a href="http://www.amazon.com/Falling-Upward-Spirituality-Halves-Life/dp/0470907754">Falling Upward</a></u> by Richard Rohr and got to this quote:<br />
<blockquote class="tr_bq">
We all try to do what seems like the task that life first hands us: establishing an identity, a home, relationships, friends, community, security, and building a proper platform for our only life. But <b>it takes us much longer to discover “the task within the task,” </b>as I like to call it:<b> what we are really doing when we are doing what we are doing</b>. Two people can have the same job description, and one is holding a subtle or not-so-subtle life energy (eros) in doing his or her job, while another is holding a subtle or not-so-subtle negative energy (thanatos) while doing the exact same job. ...<br />
... <b>In any situation, your taking or giving of energy is what you are actually doing</b>. Everybody can feel, suffer, or enjoy the difference, but few can exactly say what it is that is happening. Why do I feel drawn or repelled? What we all desire and need from one another, of course, is that life energy called eros! It always draws, creates, and connects things. </blockquote>
It got me to thinking about my current focus on leading teams at work and the, ah hem, "difficulties" I have had with both myself and others in this space. Thinking for a marketing guy inevitably ends up with a quad chart being produced... so here it is...<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-5C6hRHsV12GNFr5Lftr579SZGzr399nntPcjW-hXyLGV_PWPELatCGDG9UcxFdodttBs34e8aYYRprixvcpRLyXCubejDtkH-ddAPsRJHzqDpM5pTIJ1X1FqVS5dMoD_OIPLXr2INdEi/s1600/Energy+vs+Focus.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-5C6hRHsV12GNFr5Lftr579SZGzr399nntPcjW-hXyLGV_PWPELatCGDG9UcxFdodttBs34e8aYYRprixvcpRLyXCubejDtkH-ddAPsRJHzqDpM5pTIJ1X1FqVS5dMoD_OIPLXr2INdEi/s1600/Energy+vs+Focus.PNG" height="295" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
Have you met these people?<br />
<ul>
<li>The <b>depressive </b>energy vampire who manages to get nothing done because of all the "impossible" problems? Though he never takes the team down, he isn't particularly inspiring to be around.</li>
<li>The depressives' opposite, the <b>zealot</b>, who is unshakable and, for better or for worse, is unstoppable? Though he may scout out and create amazing bright spots he can't bring the team along with him.</li>
<li>The <b>saboteur </b>who manages to poison the well with action blocking negativity? This guy is the most dangerous because he does not stop with himself. He's especially dangerous if he is in a position of significant organizational power.</li>
<li>The <b>leader </b>who infuses the right people with the right bits of encouragement when and where they are needed to let the team find its own way to success?</li>
</ul>
Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-87322637989859225952013-12-22T17:23:00.000-08:002013-12-22T17:23:43.191-08:00The Essence of Big Data <div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjt514BjujCE2ssylv5ZP0QuzJRnopJv0D8ZYLZfpN0kFrBgYrt55VaGvj8UH3iSHoNVN92op0nBzM4Zc5uvSgsE7eTtX90XaJz7RCyaJg4CHrwvfVMDdWf2l-q3fn_tNvBfd5lkNsuSByk/s1600/Essence+of+big+data+diagram.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjt514BjujCE2ssylv5ZP0QuzJRnopJv0D8ZYLZfpN0kFrBgYrt55VaGvj8UH3iSHoNVN92op0nBzM4Zc5uvSgsE7eTtX90XaJz7RCyaJg4CHrwvfVMDdWf2l-q3fn_tNvBfd5lkNsuSByk/s1600/Essence+of+big+data+diagram.png" height="243" width="400" /></a></div>
It seems like the term "Big Data" has come to refer to so many things that it has become more of an aspirational marketing term than a technical one.<br />
In dealing with "big data" projects at work I have come to a definition that seems useful.<br />
<br />
Big Data is composed of several systems that work together:<br />
<br />
<ul>
<li>The <b>hardware that enables all the other algorithms</b> needed to handle and analyze the data of interest.</li>
<li>The <b>algorithms which efficiently load, transform and store data</b> from varied, high volume data flows.</li>
<li>The <b>algorithms which retrieve and manage the stored data</b> in a way that allows other algorithms and s/w tools to act on it efficiently.</li>
<li><b>Correlation algorithms</b> which automatically comb through the data looking for data items which are related in ways which might be "interesting" to the end user.</li>
<li>The <b>visualization tools</b> which are used to look at the automatically flagged "interesting" subset of data in order to determine if the data is actually useful and if so, how.</li>
<li>The <b>actionable information that the end user extracts</b> from the system which he uses to further whatever goals originally justified implementing the big data system.</li>
</ul>
<div>
The hardware, ETL and data storage parts have been addressed by a fairly large number of vendors using proprietary and open source methods. You could say that the data handling platform is becoming a commodity because the movement of data is an undifferentiated requirement for all big data users.</div>
<div>
<br /></div>
<div>
What is still hard is algorithmically filtering the flood of incoming data to pull out the nuggets of interest so that someone can confirm their meaning. The aspects which qualify something as interesting differ from industry to industry and company to company so coming up with a common, turn-key solution may not be possible. So, until that statement is proven false, the seller's market for data scientists will continue.</div>
<div>
<br /></div>
<div>
<br /></div>
Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-63248155461728518232013-07-23T23:53:00.000-07:002013-07-23T23:53:18.813-07:00Mixing Metaphors: BAC! Who left this MES?<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiV4GzZsgJzW5vPe4C8bzdUV3ZrbtoaqFP5dviXPGPzHytHMvsB9TwiusEYhgryWDOl0bxowsIMs6DMHINkhiYy1y4SHMCcHLZznsWH2GVENpGNCrBxTlWjKlVBQ4A5DvxGwkL925ggJt0t/s1600/building+vs+wafer.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiV4GzZsgJzW5vPe4C8bzdUV3ZrbtoaqFP5dviXPGPzHytHMvsB9TwiusEYhgryWDOl0bxowsIMs6DMHINkhiYy1y4SHMCcHLZznsWH2GVENpGNCrBxTlWjKlVBQ4A5DvxGwkL925ggJt0t/s1600/building+vs+wafer.png" height="213" style="cursor: move;" width="320" /></a>I have been learning a bit about <a href="http://en.wikipedia.org/wiki/Building_management_system">building management systems (BMS)</a> and realized that, abstracted the right way, the <a href="http://en.wikipedia.org/wiki/Manufacturing_execution_system">manufacturing execution systems (MES)</a> of semiconductor fabs are pretty similar in overall concept and components. That parallel made it much easier for me to frame my learning. </div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>The brains of the system: BMS vs MES.</b></div>
<div class="separator" style="clear: both; text-align: left;">
These s/w systems take schedules, control targets and feedback to make the building "work" (literally).</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbyy3ZET3ZyTcmdmRq5cw_L8WgqbKcTXIAqZYTQ8IbFgY3iXWVPejNlhjyRItjoEQHPGWOWDTd06gHBvxKUWmkhWBN_ezAqcEQyixNCKcZLQKatrVMvMf62vrdroL526E2vJz6skMjNdJX/s1600/BMS+vs+MES.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbyy3ZET3ZyTcmdmRq5cw_L8WgqbKcTXIAqZYTQ8IbFgY3iXWVPejNlhjyRItjoEQHPGWOWDTd06gHBvxKUWmkhWBN_ezAqcEQyixNCKcZLQKatrVMvMf62vrdroL526E2vJz6skMjNdJX/s1600/BMS+vs+MES.png" height="196" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>The communication protocols: <a href="http://en.wikipedia.org/wiki/BACnet">BACnet</a> vs <a href="http://en.wikipedia.org/wiki/SECS/GEM">SECS/GEM</a>.</b></div>
<div class="separator" style="clear: both; text-align: left;">
Granted, building equipment use many more protocols than just BACnet, some of which are proprietary to the equipment vendor.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAWhAaj6YtbcJLs4UiK_4257Hn38cDEfQk3HxMEVl_kSWTnsDhV0_7HNhZ48XQ51l9gpqKGI8SfrIdYGhi7_wdoORTzP_MriTlQxdRYzogd9kRzrGFXAXX02trigQ9UNO-r-QQiyj9Jy9O/s1600/BACnet+vs+SECS-GEM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAWhAaj6YtbcJLs4UiK_4257Hn38cDEfQk3HxMEVl_kSWTnsDhV0_7HNhZ48XQ51l9gpqKGI8SfrIdYGhi7_wdoORTzP_MriTlQxdRYzogd9kRzrGFXAXX02trigQ9UNO-r-QQiyj9Jy9O/s1600/BACnet+vs+SECS-GEM.png" /></a></div>
<br />
<br />
<b>The building equipment which is actually being controlled for the purpose of making the building useful for its owners. </b><br />
An office building could be thought of a making an environment conducive to worker productivity with maximum efficiency.<br />
A fab could be thought of as controlling the flow of materials between equipment to maximize output of wafers/chips at minimum cost.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh27AyErsGZnG7liW2a6JINrG6oecrpone1KmJx_IBvI6e4cEF3EayRs8TL2uChKthxsWe1gGSt44dHc5gutATE-Wa6l-j5mN4oILH7fhOuhn5UH9ScvpnBcPRVgMyGVqaObbtALS6Az71Q/s1600/HVAC+vs+Metrology.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh27AyErsGZnG7liW2a6JINrG6oecrpone1KmJx_IBvI6e4cEF3EayRs8TL2uChKthxsWe1gGSt44dHc5gutATE-Wa6l-j5mN4oILH7fhOuhn5UH9ScvpnBcPRVgMyGVqaObbtALS6Az71Q/s1600/HVAC+vs+Metrology.png" height="203" width="640" /></a></div>
<br />
<b>The sensor systems which allow the control system to make smart choices about controlling the building equipment.</b><br />
For an office building this might mean feedback control for HVAC (don't over cool) and switching off unused lights or dimming lights for daylight harvesting.<br />
For a fab this might mean monitoring the voltage and flow rates for a particular piece of process equipment and adjusting the process recipe for the next lot or wafer to ensure uniform film properties from lot to lot.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgcYai6xjo1WL-D9S3y9jjqJ6usVjAldOrAJ-nOp9B1Zp57yJiZiKWC3g_ofMJr8Pd4YN2YPVge53dlOK8Tc4obrQoVTP19dgBM9W-QabZahW6W0OHhV0jf1ahk-njRP-LDHzvkuszsPGUc/s1600/building+sensors+vs+embedded+sensors.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgcYai6xjo1WL-D9S3y9jjqJ6usVjAldOrAJ-nOp9B1Zp57yJiZiKWC3g_ofMJr8Pd4YN2YPVge53dlOK8Tc4obrQoVTP19dgBM9W-QabZahW6W0OHhV0jf1ahk-njRP-LDHzvkuszsPGUc/s1600/building+sensors+vs+embedded+sensors.png" height="173" width="640" /></a></div>
Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-61708697130331293282013-03-08T23:50:00.000-08:002013-03-09T00:03:27.981-08:00Wait... There's a Genie in my Economy!<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxjHAPWPrbzVBPdCj8V5uVnm6jAAcyTK-ZDpGlaGlCijTCs8x8XQ5b1j22citep5azyz_n7wlkU_vRIkmaG3ta5jjZXMJY8LLUhga0QCvFUkgdPcKrx2c2N7i1AIFh6S3uIiU-ug7OllH-/s1600/genie_1.jpg" imageanchor="1" style="clear: right; display: inline !important; float: right; margin-bottom: 1em; margin-left: 1em; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxjHAPWPrbzVBPdCj8V5uVnm6jAAcyTK-ZDpGlaGlCijTCs8x8XQ5b1j22citep5azyz_n7wlkU_vRIkmaG3ta5jjZXMJY8LLUhga0QCvFUkgdPcKrx2c2N7i1AIFh6S3uIiU-ug7OllH-/s1600/genie_1.jpg" height="320" width="240" /></a><br />
What do the following have in common?<br />
<ul>
<li>International Paper</li>
<li>United Airlines</li>
<li>Google</li>
<li>Facebook</li>
<li>Twitter</li>
<li>Amazon EC2</li>
<li>Kickstarter</li>
<li>3D Printing</li>
<li>Genies</li>
</ul>
<div>
Friction.</div>
<div>
<br /></div>
<div>
More specifically, they each help to reduce the friction between all of the things that must be in place to get something that you want.</div>
<div>
<ul>
<li>International Paper - What does it take to get the paper in your printer? </li>
<ul>
<li>Rights to log trees. The labor to log them. The tools to cut down a tree. The knowledge to use those tools. The transports to move the trees. The machines to pulp the tree. The labor and knowledge to use the pulping equipment. The chemicals to process the pulp into paper... you get the idea...</li>
</ul>
<li>United Airlines - What does it take to move yourself to Japan?</li>
<ul>
<li>The money to buy an airplane. The knowledge to fly it. The contacts required and hours spent to negotiate the rights to take off from SFO and land at NRT. The labor and knowledge to service the aircraft... etc...</li>
</ul>
<li>Google - What does it take to find out about everything on the internet?</li>
<ul>
<li>The knowledge to create an algorithm that is helpful at finding what you want amid tons of stuff you don't. The programming skills to implement it. The knowledge to build the IT infrastructure to process and store all of the data required to run the algorithm. The servers and real-estate required to hold the servers... how easy would those be to get on your own?</li>
</ul>
<li>Facebook - What does it take to find all of your long lost high school friends?</li>
<ul>
<li>The hours and hours of phone calls to numbers in your old day runner (<a href="http://www.dayrunner.com/">they still make these</a>?) hoping that their parents still remember you and still live there. Or trawling through phone directories looking for the right Joe Smith... ugh...</li>
<li>OR build your own content site which will attract half of the planet AND get them to list their high school... pretty simple...</li>
</ul>
</ul>
<div>
The remaining companies or topics flip the equation a bit as they are more general tools for reducing friction towards the end of doing something else.</div>
<ul>
<li>Twitter - How could I publish my thoughts to "everyone" at a reasonable cost?</li>
<ul>
<li>I could never mail a letter, call by phone or place enough radio and TV ads to do this. What would it cost to generate the lead list and qualify the leads to do this in a more focused way?</li>
</ul>
<li>Amazon EC2 - How do I start a s/w business that scales without major capital outlays?</li>
<ul>
<li>How else can I get enough computers to scale my SaaS business to profitability without the friction of convincing someone to front a significant amount of money to purchase and administer a server farm?</li>
</ul>
<li>Kickstarter - How do I find funding to raise capital to do something people want to see done?</li>
<ul>
<li>Am I lucky enough to be born rich? Did I get lucky enough to know powerful, rich people? Am I a good enough social engineer to find these people? Do know the right VCs? Is my product profitable enough to a VC for them to consider? What would it cost to build the audience of millions who are engaged enough to put money on the table - sight unseen?</li>
</ul>
<li>3D Printing - How do I make a complicated, custom physical part in low volume (qty 1)?</li>
<ul>
<li>The money and space to buy a CNC machine plus the experience and knowledge to operate it? Or the hours spent to find a machine shop that will do a low volume run, now, for a reasonable price?</li>
</ul>
<li>Genies - How do I do anything with anyone, anywhere at any time?</li>
<ul>
<li>You have 3 wishes...</li>
</ul>
</ul>
</div>
<div>
The interesting thing about removing the friction around doing "something else" is that it enables new ways for people to do things for themselves and, ultimately, find others who might want those things. Which they then might trade something for (like money). Which sounds sort of like an economy.</div>
<div>
Take that to its logical conclusion where friction is, genie-like, reduced to near zero between all people and the resources / skills they hold and what is the purpose of a corporation as we know it today? We could do anything for ourselves by finding and coordinating the right people.</div>
<div>
<br /></div>
<div>
Maybe this does not happen in my lifetime, but the idea of friction seems like a powerful filter for looking at the value of any product or service that you are trying to create today. If it is not reducing friction then you're heading the wrong way.</div>
<div>
<br /></div>
<div>
-----------------------------------------------------------------------------------------------------</div>
<div>
<br /></div>
<div>
Thanks to Gabe Newell for throwing the lightning bolt which fused the 10,000 threads in my brain into a coherent idea.</div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/t8QEOBgLBQU?feature=player_embedded' frameborder='0'></iframe></div>
<div>
<br /></div>
<div>
Take a look at this talk if you have an hour to spare.</div>
<div>
Why is Valve structured as it is?</div>
<div>
What is the purpose of a corporation in recent history and today?</div>
<div>
<br /></div>
<div>
Good stuff. </div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-84519167717001133942012-09-23T00:00:00.001-07:002012-09-23T00:00:28.255-07:00InfoSphere Big Data - A Cursory OverviewAfter a half day of talking with IBM reps about <a href="http://www-01.ibm.com/software/data/infosphere/">Big Data products</a> and some use cases, here is how I summarized how the pieces fit together.<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiN49MifatmGhDFdVfkQozZlKEOjDR-FqGdzPisQo6clmdXsaTo5kgiqR10krr4WTTX6ZBZeQ4T-zaLMAdOHoznQAZnUa8JXLkOpYpbMpACOSdXDErgl7cW3PAp3anyO4y6rT89YpYP-5En/s1600/IBM+InfoSphere+EcoSystem+Diagram+(w-feedback).png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="476" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiN49MifatmGhDFdVfkQozZlKEOjDR-FqGdzPisQo6clmdXsaTo5kgiqR10krr4WTTX6ZBZeQ4T-zaLMAdOHoznQAZnUa8JXLkOpYpbMpACOSdXDErgl7cW3PAp3anyO4y6rT89YpYP-5En/s640/IBM+InfoSphere+EcoSystem+Diagram+(w-feedback).png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
At time 0 you collect everything and analyze it for correlations to determine which data items are valuable and how they relate to each other (Big Insights platform). Then you build a control model.<br />
<br />
Learning from time 0 is used to configure a "real time" strategy for the data analysis and system control.<br />
<br />
<ul>
<li>Streams provide real time processing of data "on the wire" - nothing need be stored. The output of this is three fold:</li>
<ul>
<li>"Live" reports for users</li>
<li>A data subset to feed to the data warehouse</li>
<li>Control signals to feed back to the data collectors to adjust behavior (if needed).</li>
</ul>
<li>Netezza (Data Warehouse) provides a location where "fast" analysis on a "limited" subset of the data can occur.</li>
</ul>
Hadoop holds everything else so that longer term analysis with full data sets is possible. This could be used to:<br />
<br />
<ul>
<li>Adjust the control models</li>
<li>Change which data subsets are warehoused </li>
<li>Perform ad hoc deep dive analysis. </li>
<li>Perform regular analysis on data sets which are too large to reasonably warehouse (e.g. raw scan data).</li>
</ul>
Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-27896827219180784612012-08-31T23:18:00.000-07:002012-08-31T23:18:42.006-07:00Yes! We Have NoSQL Today.<div class="separator" style="clear: both; text-align: left;">
Once you have a system which can process huge amounts of data (<a href="http://cursoryknowledge.blogspot.com/2012/06/little-bit-about-big-data.html">big data</a>), you need a place to store all of that data. This is what databases are for.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Traditionally, this has meant a relational database. But relational databases place many constraints on how the data is modeled ("<a href="http://databases.about.com/od/specificproducts/a/normalization.htm">normal forms</a>") which are inconsistent with the high volume data sources which need to be analyzed (e.g. all the webpages in the world, all the legal documents in your company or all the tweets being posted each day).</div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<ul>
<li>Relational DBs require that data be modeled into a set of tables that contain unique entities (rows), described by attributes (columns) which are arranged in such a way as to describe one aspect of each entity in each table with no redundancies.</li>
<ul>
<li>said another way: </li>
<ul>
<li>each row has a primary key made from one or more columns. Column data contains single values (1NF).</li>
<li>All columns in a table relate only to the <b>complete </b>primary key (2NF)</li>
<li>All columns in the table contain data which is not derived from other columns in that table (3NF)</li>
<li>To add more columns which do not fit these constraints, you must put them in another table and join them together.</li>
</ul>
</ul>
</ul>
A good example of what this means is <a href="http://stackoverflow.com/questions/723998/can-someone-please-give-an-example-of-1nf-2nf-and-3nf-in-plain-english">here</a>.<br />
<ul><ul>
<li>said yet another way: The key, the whole key and nothing but the key.</li>
</ul>
</ul>
These restrictions allow for optimal query structuring and performance while minimizing anomalies due to data changes. However they do not easily support the lack of simple structure between the contents of many data sets.<br />
<ul>
<li>Non-Relational (NoSQL) DBs remove the restrictions on data normalization and focus, instead, on optimizing around data that does not fit well into the normalized structure which relational DBs (mostly) require. Because there are different analyses of interest and different data sources which "best" embody the data of interest, there are different types of NoSQL databases.</li>
</ul>
<br />
<div class="separator" style="clear: both; text-align: left;">
Below is a diagram showing the various database types.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGGnUK5HewWY1gTMpPTXMnRlEIReqh9MPY5ybd5d3yKK3-mEA_cpBKcHki4eT3LpRLdY3t8jU4kA33JCf94VzsoRdW9eQKlz9UGYJQkUIL8I4LN_nQf2ayBlBZ4TAcvDadE5PCdw6w1qIW/s1600/NoSQL+databases.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGGnUK5HewWY1gTMpPTXMnRlEIReqh9MPY5ybd5d3yKK3-mEA_cpBKcHki4eT3LpRLdY3t8jU4kA33JCf94VzsoRdW9eQKlz9UGYJQkUIL8I4LN_nQf2ayBlBZ4TAcvDadE5PCdw6w1qIW/s640/NoSQL+databases.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<ul>
<li>Key Value (aka Big Table)</li>
<ul>
<li>Data is stored in a GIANT ordered table of rows and columns. </li>
<ul>
<li>Rows and columns still serve the same general purpose as in a relational DB case</li>
<ul>
<li>rows = unique entities</li>
<li>columns = attributes.</li>
<li>...but normalization is not required (or expected)...</li>
</ul>
</ul>
<li>Data may be sparsely populated in the columns. </li>
<ul>
<li>I.e. a given row may only have data values for a small fraction of the columns (because most the columns don't apply to the entity this row describes). </li>
<li>Columns may be VERY large in number and depend on what the DB is structured to query for.</li>
<ul>
<li>e.g. all unique word pairs for the entities in the database</li>
</ul>
</ul>
<li>Google originally developed this technology for searching through web pages to fulfill search criteria. Roughly speaking:</li>
<ul>
<li>rows = web pages</li>
<li>columns = search terms</li>
</ul>
</ul>
<li>Document</li>
<ul>
<li>Entire documents are stored in a searchable format.</li>
<ul>
<li><a href="http://en.wikipedia.org/wiki/JSON">JSON </a>(JavaScript Object Notation)</li>
<li><a href="http://en.wikipedia.org/wiki/BSON">BSON</a> (Binary JavaScript Object Notation)</li>
<li><a href="http://en.wikipedia.org/wiki/XML">XML </a>(eXtensible Markup Language)</li>
</ul>
<li>Queries search through the documents to identify the information of interest and return statistics or the document IDs.</li>
<li>Good for finding actual documents which contain specific information or <a href="http://cursoryknowledge.blogspot.com/2012/04/essence-of-data-analysis.html">summarizing </a>the information contained in a set of documents. </li>
</ul>
<li>Graph</li>
<ul>
<li>Stores information about relationships between entities (objects) in the DB</li>
<li>Good for finding objects that are related to each other according to certain criteria.</li>
<ul>
<li>e.g. find people (entities) who are members of the YMCA (another entity) who lived in New York in 1999.</li>
</ul>
</ul>
</ul>
<u>How does this relate to Big Data?</u><br />
Many NoSQL DBs are built to operate on distributed file systems and process queries via distributed computing. In fact, the very nature of the data being looked at is so large<br />
<ul>
<li><a href="http://www.businessweek.com/articles/2012-08-23/facebooks-is-bigger-than-yours">operating on >100 PetaBytes and ingesting >500 TeraBytes/day for Facebook</a></li>
<li>processing <a href="http://arstechnica.com/business/2012/01/the-big-disk-drive-in-the-sky-how-the-giants-of-the-web-store-big-data/">>20 Petabytes / day for Google</a></li>
<li><a href="http://www.zdnet.com/blog/btl/twitter-turns-six-140-million-users-340-million-tweets-daily/72123">Processing > 340 Million tweets / day</a> for Twitter (~44GB/day but a huge number of entities)</li>
</ul>
and so unstructured (hard to normalize) that the data could be stored and handled in no other way.Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-42010931429621808852012-08-19T23:40:00.000-07:002012-08-19T23:40:02.136-07:00Can't Test This... A/B TestingA/B testing is a very powerful tool for developing certain kinds of products. Here are a few thoughts on where it does and doesn't work.<br />
<br />
Below is a high level flow of the testing cycle.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXhMpuMIYM0D5YLksCO-2HgNA8NSSECDYaTRVsHOsGADSm6-Ke4Xedzdb-LIDQmSEqPLuOCsxaVWd5sDmYVvT1ePVQV0OEpjhFWo5rO5P0uFQ82eP3xfKCIiVwhN-DMnvLauDUM0GegscH/s1600/AB+Testing+flowchart.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="282" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXhMpuMIYM0D5YLksCO-2HgNA8NSSECDYaTRVsHOsGADSm6-Ke4Xedzdb-LIDQmSEqPLuOCsxaVWd5sDmYVvT1ePVQV0OEpjhFWo5rO5P0uFQ82eP3xfKCIiVwhN-DMnvLauDUM0GegscH/s640/AB+Testing+flowchart.png" width="640" /></a></div>
<br />
<br />
A/B testing works when:<br />
<br />
<ul>
<li>The cost of implementing ver A & B + </li>
<li>The cost of collecting "enough" data about A&B + </li>
<li>The cost of fanning out the "best" version, </li>
</ul>
<div>
is less than:</div>
<div>
<ul>
<li>The cost of visiting "enough" of your key customers + </li>
<li>The cost of spending "enough" time with each of them to understand the full requirements.</li>
</ul>
<div>
<br /></div>
<div>
Or framed a different way...</div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikfc74lGucwGKA7mvKfnqjl2II4zZ-SqjNfWNVmS8TMcWhhQDIRmcFnjLjEZvvQXx6aCDKXpTuYMZY7J_EmOFMtyPPFuwTisgijU_93jfKG79KkwMT4M8hf4jkv5e78jo5SmteHvUb5vmg/s1600/AB+Testing+difficulty.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="488" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikfc74lGucwGKA7mvKfnqjl2II4zZ-SqjNfWNVmS8TMcWhhQDIRmcFnjLjEZvvQXx6aCDKXpTuYMZY7J_EmOFMtyPPFuwTisgijU_93jfKG79KkwMT4M8hf4jkv5e78jo5SmteHvUb5vmg/s640/AB+Testing+difficulty.png" width="640" /></a></div>
<div>
If it costs a significant amount to develop, test or deploy the thing you want to evaluate or</div>
<div>
If you cannot get adequate information back from the customer base or</div>
<div>
If you cannot get information back in a reasonable amount of time,</div>
<div>
<br /></div>
<div>
then there may be better development approaches than A/B testing.</div>
<div>
<br /></div>
Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-71496977232149619562012-08-13T23:47:00.000-07:002012-08-13T23:56:32.969-07:00The Essence of a Marketing Requirements Document (MRD)<div class="separator" style="clear: both; text-align: left;">
A few thoughts about writing MRDs.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
NOTE: The MRD should describe WHAT needs to happen overall and between parts. The MRD should not (usually) describe HOW all the parts get implemented - that is for the engineering design document.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEyTepW0cej3LSHxHL7au9s3uEJrMzYhCtUKvyzxSYSwhyphenhyphenylgxZ4-VOxFwTqOCoCZ43aJpCwtKezVCYublCgDBK63n00w4Eqbb4dfd3xTQo5GbLeMRfI568rToptnL0BGRljtBnaU2Nnu-/s1600/MRD+baseblock.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="256" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEyTepW0cej3LSHxHL7au9s3uEJrMzYhCtUKvyzxSYSwhyphenhyphenylgxZ4-VOxFwTqOCoCZ43aJpCwtKezVCYublCgDBK63n00w4Eqbb4dfd3xTQo5GbLeMRfI568rToptnL0BGRljtBnaU2Nnu-/s640/MRD+baseblock.png" width="640" /></a></div>
<br />
<br />
<ol>
<li>Describe the end-to-end scope of the problem to be solved</li>
<li>Break the problem into logical sub-problems</li>
<li>Describe the inputs required to resolve each sub-problem. This includes:</li>
<ul>
<li>human interfaces for data input </li>
<ul>
<li>one time </li>
<li>interactive / iterative </li>
</ul>
<li>machine / data inputs from external data</li>
<li>machine / data inputs from internal (transient) data</li>
</ul>
<li>Describe what output should be generated by resolving each sub-problem. This includes:</li>
<ul>
<li>which data is needed as "the" output. i.e. the "permanent" data. </li>
<ul>
<li>What is the expected input format of the consumer(s)?</li>
</ul>
<li>which data is needed to address another sub-problem. i.e. transient data.</li>
</ul>
</ol>
<div>
All of this should be written with an eye to the system in which the functionality described by the MRD lives.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTZQPlr66aGcHy6X-z90RV00T3DFkd8Nm_7dNLZF7yLru4ZobyHHRtVvAmRuzy5L0tXvV3Q9ITqLFDbiInu1aNqr3sBdabQxFc1IY8OYKRcK4Zn4XZPwBs_gFpyeyAVuCUYAx2kVSIEp2I/s1600/MRD+of+the+system+%2528recursive%2529.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="638" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTZQPlr66aGcHy6X-z90RV00T3DFkd8Nm_7dNLZF7yLru4ZobyHHRtVvAmRuzy5L0tXvV3Q9ITqLFDbiInu1aNqr3sBdabQxFc1IY8OYKRcK4Zn4XZPwBs_gFpyeyAVuCUYAx2kVSIEp2I/s640/MRD+of+the+system+%2528recursive%2529.png" width="640" /></a></div>
<div>
<br /></div>
<div>
<ul>
<li>Every input is the output of another system, ideally described by an MRD (reference it if you can).</li>
<li>Other systems may need the output of the system described by your MRD. Include these systems as examples in your MRD to give color to the bigger picture problem being solved.</li>
<li>Human input interfaces (User Interfaces), Machine Input interfaces (APIs) and Permanent Data stores (HDDs or Databases) may be shared between multiple systems. If they are, or should be, note that explicitly.</li>
</ul>
<div>
One obvious challenge, given the recursive approach to MRD writing given here, is figuring out where to stop.</div>
<div>
How big should the scope of THE problem be?</div>
</div>
<div>
My experience: when in doubt, make the scope too big. Then scale back the scope during reviews based on feedback from the stakeholders.</div>
Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-38274943970670579062012-07-25T23:57:00.000-07:002012-07-25T23:57:11.986-07:00The Relationship Between Virtualization, Big Data and Cloud Computing<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7Mz2P3o_zfYnYluLHp2YVo0Wxz8NWddqFfBNbSQp9E1WCEh3I0CqdGcRA_5p3f9qhJk-GUdtX-eeVki8lntL6cuQrPkdD0K-bRBr4C58c53OABc2zTdoDWVAgdgWMdjDaslox3UWCg_cS/s1600/Virtualization.png" imageanchor="1" style="background-color: white; clear: left; display: inline !important; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" height="193" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7Mz2P3o_zfYnYluLHp2YVo0Wxz8NWddqFfBNbSQp9E1WCEh3I0CqdGcRA_5p3f9qhJk-GUdtX-eeVki8lntL6cuQrPkdD0K-bRBr4C58c53OABc2zTdoDWVAgdgWMdjDaslox3UWCg_cS/s640/Virtualization.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<b><span style="color: blue;">Virtualization </span></b>is about taking a single large compute resource and making it act like many smaller resources.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOU-EEFoKUKtFcFBmPGaP4doNoiqTVtTXgK1nq490R_8mkZHGx13DHrLYJ8IFvS1syAkx5_OVj7zkRwA2k1gFf5YaC7GqgzTDcESb8HbFV68-aXvRySSlVbqyBcoTtgqUVnUWlC1IXCgP9/s1600/horizontal+line.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="12" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOU-EEFoKUKtFcFBmPGaP4doNoiqTVtTXgK1nq490R_8mkZHGx13DHrLYJ8IFvS1syAkx5_OVj7zkRwA2k1gFf5YaC7GqgzTDcESb8HbFV68-aXvRySSlVbqyBcoTtgqUVnUWlC1IXCgP9/s640/horizontal+line.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhP-cDDetAfevyrVSCVBSEKyHNpHYTpythjyrZUoxLSip4GQHxN_m0SHiGb1-MZ0DIsAPz0n16ECHOwT4MpUVl3gPAuWAOJQm0EdP5p8hjue-7DSwkFuPBmTSOsRflQFOBc_e1VzQ3R1V_d/s1600/Big+Data.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="216" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhP-cDDetAfevyrVSCVBSEKyHNpHYTpythjyrZUoxLSip4GQHxN_m0SHiGb1-MZ0DIsAPz0n16ECHOwT4MpUVl3gPAuWAOJQm0EdP5p8hjue-7DSwkFuPBmTSOsRflQFOBc_e1VzQ3R1V_d/s640/Big+Data.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<b><span style="color: blue;">Big Data</span></b> is about taking many smaller compute (and storage) resources and making them act like one big resource.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOU-EEFoKUKtFcFBmPGaP4doNoiqTVtTXgK1nq490R_8mkZHGx13DHrLYJ8IFvS1syAkx5_OVj7zkRwA2k1gFf5YaC7GqgzTDcESb8HbFV68-aXvRySSlVbqyBcoTtgqUVnUWlC1IXCgP9/s1600/horizontal+line.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="12" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOU-EEFoKUKtFcFBmPGaP4doNoiqTVtTXgK1nq490R_8mkZHGx13DHrLYJ8IFvS1syAkx5_OVj7zkRwA2k1gFf5YaC7GqgzTDcESb8HbFV68-aXvRySSlVbqyBcoTtgqUVnUWlC1IXCgP9/s640/horizontal+line.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCSF9B8RUSBij7c_sJCsIU5TjJbVJ54q114Z0Wg6Zof5y4vcbz0CTnElD8k59rcGY8eVMyiNHkjGR5zqmhdTHhI5HdMCCJ0s6SUU3TgC7sz3Pk2SR5C16l4MSTU7swgr-no1Zj8LN-z8zj/s1600/Cloud+computing.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="398" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCSF9B8RUSBij7c_sJCsIU5TjJbVJ54q114Z0Wg6Zof5y4vcbz0CTnElD8k59rcGY8eVMyiNHkjGR5zqmhdTHhI5HdMCCJ0s6SUU3TgC7sz3Pk2SR5C16l4MSTU7swgr-no1Zj8LN-z8zj/s640/Cloud+computing.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<b><span style="color: blue;">Cloud computing</span></b> is about easily changing my compute and storage resources as needed.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOU-EEFoKUKtFcFBmPGaP4doNoiqTVtTXgK1nq490R_8mkZHGx13DHrLYJ8IFvS1syAkx5_OVj7zkRwA2k1gFf5YaC7GqgzTDcESb8HbFV68-aXvRySSlVbqyBcoTtgqUVnUWlC1IXCgP9/s1600/horizontal+line.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="12" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOU-EEFoKUKtFcFBmPGaP4doNoiqTVtTXgK1nq490R_8mkZHGx13DHrLYJ8IFvS1syAkx5_OVj7zkRwA2k1gFf5YaC7GqgzTDcESb8HbFV68-aXvRySSlVbqyBcoTtgqUVnUWlC1IXCgP9/s640/horizontal+line.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiH4yhOrnBFC0R3fJka1yk163YY0xuwhe9IK7emoEtWcPwv2EQOuHHJp5GT9_-zBe0UqgYQ1eKrrnM9WUYWmbzNVrk1lerCdRr_08b5G-4kE4gMv1iP2cBc6UvjwmKOe_0Br0rB54SbQSpa/s1600/big+data+in+the+cloud.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="286" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiH4yhOrnBFC0R3fJka1yk163YY0xuwhe9IK7emoEtWcPwv2EQOuHHJp5GT9_-zBe0UqgYQ1eKrrnM9WUYWmbzNVrk1lerCdRr_08b5G-4kE4gMv1iP2cBc6UvjwmKOe_0Br0rB54SbQSpa/s640/big+data+in+the+cloud.png" width="640" /></a></div>
<div style="text-align: left;">
<b style="background-color: white;"><span style="color: blue;"><br /></span></b></div>
<div style="text-align: left;">
<b style="background-color: white;"><span style="color: blue;"><br /></span></b></div>
<div style="text-align: left;">
<b style="background-color: white;"><span style="color: blue;"><br /></span></b></div>
<div style="text-align: left;">
<b style="background-color: white;"><span style="color: blue;"><br /></span></b></div>
<div style="text-align: left;">
<b style="background-color: white;"><span style="color: blue;"><br /></span></b></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<b style="background-color: white;"><span style="color: blue;"><br /></span></b></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<span style="color: blue;"><b><br /></b></span></div>
<div style="text-align: left;">
<b style="background-color: white;"><span style="color: blue;"><br /></span></b></div>
<div style="text-align: left;">
<b style="background-color: white;"><span style="color: blue;"><br /></span></b></div>
<div style="text-align: left;">
<b style="background-color: white;"><span style="color: blue;"><br /></span></b></div>
<div style="text-align: left;">
<b style="background-color: white;"><span style="color: blue;"><br /></span></b></div>
<div style="text-align: left;">
<b style="background-color: white;"><span style="color: blue;"><br /></span></b></div>
<div style="text-align: left;">
<b style="background-color: white;"><span style="color: blue;"><br /></span></b></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<b style="background-color: white;"><span style="color: blue;">Big Data</span></b><span style="background-color: white;"> can </span><u style="background-color: white;">leverage </u><b style="background-color: white;"><span style="color: blue;">cloud computing</span></b><span style="background-color: white;"> to scale the size of the "one big" resource as needed.</span></div>
<div style="text-align: left;">
<span style="background-color: white;"><br /></span></div>Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-22707815206341340872012-07-07T00:08:00.001-07:002012-07-07T00:08:35.713-07:00The Essence of VisionOne of the best summaries I have seen of what implementing one's vision means.<br />
<br />
<ul>
<li><span style="background-color: white;">Experience many things in order to distill your vision</span></li>
<li><span style="background-color: white;">Make it your mission</span></li>
<li><span style="background-color: white;">Reduce it to a question</span></li>
<li><span style="background-color: white;">Apply the question relentlessly to your actions.</span></li>
</ul>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9vMNWtnjG91hi-hs4lwxs9sqPrNa97PAR_UO5nSZSgaaGJIKj_ZM4gR6LCHcgWkW-fJkrWLwdY8Cs4Hn7h6i798c0h30owoDnoawumpfo4Y3RHyujCbtg1xOI_L_wcShvK-wXwS9q-hn6/s1600/Essence+of+Vision+diagram.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="218" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9vMNWtnjG91hi-hs4lwxs9sqPrNa97PAR_UO5nSZSgaaGJIKj_ZM4gR6LCHcgWkW-fJkrWLwdY8Cs4Hn7h6i798c0h30owoDnoawumpfo4Y3RHyujCbtg1xOI_L_wcShvK-wXwS9q-hn6/s640/Essence+of+Vision+diagram.png" width="640" /></a></div>
<br />
<span style="background-color: white;">Quote extracted from the talk at </span><a href="http://www.webmonkey.com/2012/02/video-inventing-on-principle/" style="background-color: white;">this link</a><span style="background-color: white;">.</span><br />
<iframe allowfullscreen="" frameborder="0" height="281" mozallowfullscreen="" src="http://player.vimeo.com/video/36579366?byline=0" webkitallowfullscreen="" width="500"></iframe> <br />
<a href="http://vimeo.com/36579366">Bret Victor - Inventing on Principle</a> from <a href="http://vimeo.com/cusec">CUSEC</a> on <a href="http://vimeo.com/">Vimeo</a>.Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-12120493540815828062012-06-27T23:54:00.000-07:002012-06-27T23:59:27.463-07:00A little bit about Big Data - Hadoop<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNQdIARumHHJDkviD10RtNgwOAnbqS3zdUSh9Ffn98ecX3M8zYYUbeY75C0ININKlnmiu5vqJ9Mrov5RXzZCFp9X4_ZzJnBul7zytqaoU5ExAQ2qe8Ts5Om8XrAfJOm2ascPTOR8CUxmbK/s1600/hadoop.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="150" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNQdIARumHHJDkviD10RtNgwOAnbqS3zdUSh9Ffn98ecX3M8zYYUbeY75C0ININKlnmiu5vqJ9Mrov5RXzZCFp9X4_ZzJnBul7zytqaoU5ExAQ2qe8Ts5Om8XrAfJOm2ascPTOR8CUxmbK/s200/hadoop.jpg" width="200" /></a>One of the ways that Big Data of the sort discussed in <a href="http://cursoryknowledge.blogspot.com/2012/06/little-bit-about-big-data.html">this last post</a> is implemented is with an open source technology stack called Hadoop.</div>
<br />
Hadoop consists of a two main parts:<br />
<ul>
<li>HDFS - Hadoop File System</li>
<li>MapReduce infrastructure</li>
</ul>
<div>
These allow data processing jobs to be divided among multiple nodes and then aggregated into a single result. In essence, this constructs a large, parallel computer from many smaller computers - basically the opposite of virtualization.<br />
<br />
Schematically, a Hadoop cluster looks like this:</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnBHSYhtN2X-27Si_iWBTcNe2i_UrEntfU9Hp_VcJPDzFnb6G9KMdOUvg8IEdpC8dRzyquqa5_tvbuss2-9yqa6IhzYNKKVR_SsbZL9Q3fFUD5D4ShrvBsS07qucUWNwqyS90dDe8RHEYz/s1600/Hadoop+server+&+node+diagram.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnBHSYhtN2X-27Si_iWBTcNe2i_UrEntfU9Hp_VcJPDzFnb6G9KMdOUvg8IEdpC8dRzyquqa5_tvbuss2-9yqa6IhzYNKKVR_SsbZL9Q3fFUD5D4ShrvBsS07qucUWNwqyS90dDe8RHEYz/s640/Hadoop+server+&+node+diagram.png" width="640" /></a></div>
<br /></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<u><span style="color: blue;"><b>10,000 ft view of how it works</b></span></u><br />
<br />
<ul>
<li><span style="background-color: white;"><span style="color: blue;">The Job Tracker on the master server gets a job</span></span></li>
<li><span style="background-color: white;"><span style="color: blue;">The Job Tracker breaks up the job using the map function</span></span></li>
<ul>
<li><span style="background-color: white;"><span style="color: blue;">Basic queuing ensures that any one node is not overloaded with tasks</span></span></li>
<li><span style="background-color: white;"><span style="color: blue;">The are tasks preferentially distributed to the nodes nearest the data on which the task must operate to minimize file transfer overhead*.</span></span></li>
<ul>
<li><span style="background-color: white;"><span style="color: blue;">nearest = same node as the data resides.</span></span></li>
<li><span style="background-color: white;"><span style="color: blue;">next nearest = different node but behind the same switch (so that data transfer is localized to that network segment).</span></span></li>
</ul>
</ul>
<li><span style="background-color: white;"><span style="color: blue;">The Job Tracker gets status for all tasks via the Task Trackers as they run. </span></span></li>
<ul>
<li><span style="background-color: white;"><span style="color: blue;">If a node stops reporting, the Job Tracker will redistribute that node's tasks to another node.</span></span></li>
</ul>
<li><span style="background-color: white;"><span style="color: blue;">When all tasks are complete for a job, the Job Tracker has the nodes execute the reduce function to generate a single result from the tasks' output.</span></span></li>
<li><span style="color: blue;">The final output may then be used by other applications directly or as the input to another MapReduce iteration.</span></li>
</ul>
<br />
I am certain that I missed almost every important detail in terms of the actual engineering implementation of a Hadoop cluster. But for those who just need a cursory understanding of the technology in order to make sufficient sense of what engineering is actually talking about to sanity check proposals, I hope this hits the mark.<br />
<br />
---------------------------------------------------------------------------------------------------<br />
<u>* HDFS and the awareness of data's physical location is very important in dealing with large data.</u><br />
<span style="background-color: white;">To make compute and storage capacity scale linearly with cost, the data must be spread around all the nodes in the cluster and a record kept of where all the data actually is. It can't be kept centrally or data transfer becomes a serious bottleneck to computational performance. </span><span style="background-color: white;"><i>I.e. it could take longer to move the data to a compute node over a network than it takes to actually process it.</i> </span><br />
This distribution is managed by HDFS.<br />
<br />
HDFS holds and replicates data in the system in order to minimize the chance of a bottleneck. It does this by keeping at least 3 copies of the data:<br />
<br />
<ul>
<li><span style="background-color: white;">an original</span></li>
<li><span style="background-color: white;">a copy on another node behind the same switch as the original</span></li>
<li><span style="background-color: white;">a copy on another node on a different switch from the original</span></li>
</ul>
<br />
This replication attempts to strike a balance between:<br />
<br />
<ul>
<li><span style="background-color: white;">keeping enough copies of the data to </span><span style="background-color: white;">minimize the queue size on each node (to ensure timely completion of jobs)</span><span style="background-color: white;"> and to </span><span style="background-color: white;">ensure robust execution despite failed tasks on dead nodes </span></li>
</ul>
<span style="background-color: white;">-and-</span><br />
<ul>
<li><span style="background-color: white;">reducing the total storage capacity of the system by duplicating data.</span></li>
</ul>
</div>Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-32466117113198328692012-06-21T00:25:00.001-07:002012-06-27T23:57:26.167-07:00A little bit about Big DataThe inspection tools that I work with are capable of churning out enormous amounts of data - on the order of terabytes an hour. To handle that data volume, we have done what every company did (up until now) which was sample from the data and reformat it to fit into a gigabyte sized database. This lets the data be accessed for useful analyses but creates a problem in that much of the data is actually lost, ultimately limiting what can be learned.<br />
<br />
This is traditional data processing.<br />
<br />
To store and handle more data, we swap out the existing hardware with bigger (read: more expensive) hardware. This works only up to a point as the cost of bigger h/w does not rise linearly with capacity. So you reach a limit to what is cost effective pretty quickly.<br />
<br />
Schematically it looks something like this:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwBxvoBQDFzokDENtXlKnCfS3wnODCyCsLdUuWMqrF_yVG3S6gHPiUuj8PEssJSIKtuEvnmrpbmd0vKeknJTDwv0LbStIgWImRYUgqtarhBORtDWMQd9pqhhgZx-jym830DrGMEek2KowL/s1600/Traditional+data+diagram.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="276" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwBxvoBQDFzokDENtXlKnCfS3wnODCyCsLdUuWMqrF_yVG3S6gHPiUuj8PEssJSIKtuEvnmrpbmd0vKeknJTDwv0LbStIgWImRYUgqtarhBORtDWMQd9pqhhgZx-jym830DrGMEek2KowL/s640/Traditional+data+diagram.png" width="640" /></a></div>
<br />
<i>Several data sources structure the data and put it into a database. Programs running on the compute resources access the data from the database and provide some analysis. Scaling the system means getting bigger h/w.</i><br />
<br />
Big Data changes how this can be done.<br />
<br />
At its heart, Big Data is about making the data storage size and computing power scale in a linear way with cost. This is done using a few technologies which I will <span id="goog_19310206"></span><a href="http://cursoryknowledge.blogspot.com/2012/06/little-bit-about-big-data-hadoop.html">describe in more detail late<span id="goog_19310207"></span>r</a>.<br />
<br />
Schematically it looks something like this:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjenzMPG8-JHoCDu5qtgZ5zhTqYLltnClb6CsIm6jzPMyCyvqfOhIwvy0V2Lig6RbnNC8L_VHjR4d9-vDY6WhZrf7Sgkt2g7BsmE-IkCJ7cAwLh60VeKrq1G-KO3-RpdB_Q1-OdNbY-rvN-/s1600/Big+data+diagram.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="322" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjenzMPG8-JHoCDu5qtgZ5zhTqYLltnClb6CsIm6jzPMyCyvqfOhIwvy0V2Lig6RbnNC8L_VHjR4d9-vDY6WhZrf7Sgkt2g7BsmE-IkCJ7cAwLh60VeKrq1G-KO3-RpdB_Q1-OdNbY-rvN-/s640/Big+data+diagram.png" width="640" /></a></div>
<br />
<i>One system coordinates the actions of many nodes in order to generate a desired computing result. Each node contains both compute and storage.</i><br />
<i>The entire system works in the same basic way regardless of how many nodes are present. So if more data storage or more computing power is required it can be added by <b>provisioning more nodes instead of replacing the entire system with larger nodes</b>. This makes it easy for a company to scale its costs with actual business volume or to handle burst loads via a hybrid cloud approach (i.e. provisioning additional nodes on demand as an<a href="http://cursoryknowledge.blogspot.com/2012/02/cloud-computing-virtualization-iaas.html"> IaaS offering</a>) to avoid large capital expenditures due to over provisioning.</i><br />
<br />
Through this architecture, Big Data brings significant change to the limits of how much data can be handled in a timely manner. <a href="http://www-01.ibm.com/software/data/bigdata/">IBM has a great summary of this principle in its "three Vs</a>".<br />
<br />
<ol>
<li>Volume : Petabytes instead of terabytes.</li>
<li>Velocity : Analyzed in seconds rather than in minutes, hours or days.</li>
<li>Variety : Coming from many sources, including unstructured data sources (i.e. things that don't fit into a relational database very well).</li>
</ol>
<div>
So, instead of throwing away most of the inspector data as we do today, we could keep the data and build a system at reasonable cost which could actually process it. With hard work on new algorithms which could take advantage of the new data would come new insights into the phenomena behind the data.</div>
<div>
Not trivial but newly possible.</div>
<br />
<br />
<br />
<br />Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-33262703282748797852012-06-08T00:08:00.000-07:002012-06-08T00:08:45.625-07:00What is Agile Development an Answer to?I got to thinking about some of the problems I see at work around software development & roadmap and decided to apply some systems thinking to the situation. This is the result.<br />
<br />
<iframe allowfullscreen="true" frameborder="0" height="540" mozallowfullscreen="true" src="https://docs.google.com/presentation/embed?id=11DdrZpvx4DzIkttgU9twpK0YPVOUqCXp4e1qa_H9qFU&start=false&loop=false&delayms=15000" webkitallowfullscreen="true" width="720"></iframe><br />
<br />
The key learning:<br />
<br />
<ul>
<li>Really good use case validation is probably the largest leverage point.</li>
<li>Agile development can be an alternative to really good use case validation.</li>
</ul>
<div>
No surprises there but it is interesting to see the dynamics that lead to those conclusions. The feedback loops suggest alternative paths to address the customer acceptance problem when neither use case validation improvements nor agile development are feasible. For example:</div>
<div>
<ul>
<li>What if you refused to add late features and managed the initial urgency to gain product acceptance? As long as the gaps are fixed in the medium term, the improved roadmap credibility may be enough to gain acceptance in the face of gaps next time because the customer believes your roadmap claims.</li>
<li>If apps and product managers are failing to validate use cases sufficiently, can you increase scrutiny on requirements by engineering and increase insistence on complete test case details by SQC to minimize factors which cause schedule slips and perhaps offset the slips caused by feature adds?</li>
</ul>
<div>
<br /></div>
</div>Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-54740261313179620032012-05-10T23:15:00.000-07:002012-05-10T23:15:46.139-07:00Google Marketing - A Short Analysis<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7C-nIR6iFvWszzOkMq1Is16Ogfgz-vM8Lisg6xrg5acU3ToMwkPXYQXvgiucRFMrjcom-vxxWP-8Dwvt3wugE9JDU1Ioy4RkZh3zLEELapz8qtDEFnEA-FTi3shwB-jZgTggZ1Yv3jHk-/s1600/Google+logo.JPG" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="113" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7C-nIR6iFvWszzOkMq1Is16Ogfgz-vM8Lisg6xrg5acU3ToMwkPXYQXvgiucRFMrjcom-vxxWP-8Dwvt3wugE9JDU1Ioy4RkZh3zLEELapz8qtDEFnEA-FTi3shwB-jZgTggZ1Yv3jHk-/s320/Google+logo.JPG" width="320" /></a></div>
This is what happens when you dedicate a few hours to intensively trying to answer a single question about a single company (in a slightly modified version to remove work related info).<br />
<div>
<br /></div>
<div>
Thanks to Kwok Ng for his help on this.</div>
<br />
<br />
<br />
<center>
<iframe allowfullscreen="true" frameborder="0" height="389" mozallowfullscreen="true" src="https://docs.google.com/presentation/embed?id=1SZZebfioN2aLnvyY_L4Sv0HrlMNNje9E04dv1ujpDcg&start=false&loop=false&delayms=3000" webkitallowfullscreen="true" width="480"></iframe>
</center>
By the way... Does anyone know a GOOD way to get PPT slides onto the web with no conversion artifacts?
Neither SlideShare nor Google Docs did it for me.Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-76379955443456321332012-04-30T23:41:00.000-07:002012-04-30T23:41:13.252-07:00The Essence of Data Analysis<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKI1bVVyOSrBANujTzwf3OL_SRWhoVvxzo7VPNbRxM-XQNkb4NFaEJ2e67So8tlrgQU7wOTVsDeHsnfGxczkXyYuoROaP4QWNrQN16DMM8tLXfbnqcYILRFPzR5n09JhJ6K8PeM4QTVpBx/s1600/Analysis+schematic.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="313" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKI1bVVyOSrBANujTzwf3OL_SRWhoVvxzo7VPNbRxM-XQNkb4NFaEJ2e67So8tlrgQU7wOTVsDeHsnfGxczkXyYuoROaP4QWNrQN16DMM8tLXfbnqcYILRFPzR5n09JhJ6K8PeM4QTVpBx/s640/Analysis+schematic.png" width="640" /></a></div>Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-8242263901679781062012-02-04T23:03:00.000-08:002012-02-04T23:03:00.905-08:00Cloud Computing, Virtualization, IaaS, PaaS and SaaS - Part 2<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJsEjwQOkHqpvT7mOzgQnoKQQZszkjyLsTfJl77asNFHudFTqC-feysdTmkgN1M4lyj1h0lQweH3LxLZGgj5EdEpXxDQJLfKLQLFqHGGVKZhTySpnK0kUGWwmneDdfJyWbtjK3olAYLGvC/s1600/Cloud+computing+pyramid.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJsEjwQOkHqpvT7mOzgQnoKQQZszkjyLsTfJl77asNFHudFTqC-feysdTmkgN1M4lyj1h0lQweH3LxLZGgj5EdEpXxDQJLfKLQLFqHGGVKZhTySpnK0kUGWwmneDdfJyWbtjK3olAYLGvC/s320/Cloud+computing+pyramid.png" width="320" /></a></div>In<a href="http://cursoryknowledge.blogspot.com/2012/01/cloud-computing-virtualization-iaas.html"> part one of this post</a>, I looked in to the question of why cloud computing is important. In this post, I will look into the question of what cloud computing is.<br />
<br />
<a href="http://www.nist.gov/itl/csd/cloud-102511.cfm">NIST's definition</a> of cloud computing gives a useful model for deciding what is and is not a cloud deployment:<br />
<i>"cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction</i>."<br />
<br />
But to understand what "the cloud" is really made up of, it helps to look at how cloud computing is packaged and sold.<br />
<br />
<ul><li>Infrastructure as a Service (IaaS)</li>
<li>Platform as a Service (PaaS)</li>
<li>Software as a Service (SaaS)</li>
</ul><br />
<b>Infrastructure as a Service (Iaas)</b><br />
In this cloud offering, the seller provisions virtualized compute, network and storage upon which the buyer installs his own operating system and software. <b>This is most easily thought of as buying bare computers and storage onto which your own IT dept will put everything required for your specific uses.</b><br />
A few major vendors who offer these products are: Amazon Web Services, RackSpace and AT&T Synaptic Hosting.<br />
<br />
Virtualization software is key to making IaaS feasible. Without virtualization, physical hardware would need to be provisioned for each customer. This would make it prohibitively expensive to quickly provision, re-provision and scale resources to meet customer needs.<br />
The major virtualization s/w vendors include VMWare (ESX), Microsoft (HyperV) and an open source virtualization solution Xen.<br />
<br />
More on virtualization in another post.<br />
<br />
<b>Platform as a Service (Paas)</b><br />
In the platform level cloud offering, the seller provisions some combination of operating system, <a href="http://en.wikipedia.org/wiki/Middleware">middleware </a>and <a href="http://en.wikipedia.org/wiki/Run-time_system">runtime </a>packages which enable the buyer to develop and run applications of their choosing. <b>This can be thought of as buying access to services which enable programs to easily access the resources they need to run.</b><br />
Vendors provide a variety of offerings in this space. A few major offerings are below:<br />
<br />
<ul><li>Google App Engine provides a number of runtime environments for web application developers.</li>
<li>Amazon Simple Storage Service (S3) provides file based storage to any application which needs it while Amazon Elastic Block Store (EBS) provides block level storage for applications the need direct storage access (like DBs).</li>
<li>Microsoft Azure provides access to a few runtime envrionments, SQL DB services for applications which need DB services and the virtual <a href="http://etherealmind.com/what-is-the-definition-of-switch-fabric/">network fabric</a> required to link together multiple server's services.</li>
</ul><b>Software as a Service (Saas)</b><br />
In this level of cloud offering, the seller provides the buyer with access to end applications while hiding the infrastructure, middleware and runtime components. <b>Pretty much every time you hear about a "web application" being offered by a company, you are seeing a SaaS product.</b><br />
There are <a href="http://www.quora.com/How-many-software-companies-in-the-U-S-A-have-a-SaaS-offerings">probably thousands</a> of SaaS offerings available so covering any significant fraction of the space here is futile. But here are a few examples of SaaS offerings:<br />
<ul><li><span style="font-family: 'Times New Roman', serif; font-size: 12pt;">Web email - from google, Yahoo, etc</span></li>
<li><a href="http://www.microsoft.com/en-us/office365/online-software.aspx#fbid=gTcwNwGmSST" style="font-family: 'Times New Roman', serif; font-size: 12pt;"><span style="color: blue;">Office 365</span></a><span style="font-family: 'Times New Roman', serif; font-size: 12pt;"> and </span><a href="http://www.google.com/apps/intl/en/group/index.html" style="font-family: 'Times New Roman', serif; font-size: 12pt;"><span style="color: blue;">Google Apps</span></a><span style="font-family: 'Times New Roman', serif; font-size: 12pt;"> - On line productivity apps for creating, accessing and collaborating with others on a variety of document types (word processing, presentations, spreadsheets, picture editing, etc).</span></li>
<li><a href="http://csrware.com/ssc-supply-chain.html" style="font-family: 'Times New Roman', serif; font-size: 12pt;"><span style="color: blue;">Sustainable Supply Chain (SSC) </span></a><span style="font-family: 'Times New Roman', serif; font-size: 12pt;">- Supply chain survey management for corporate social responsibility reporting from CSRware.</span></li>
<li><a href="http://www.facebook.com/" style="font-family: 'Times New Roman', serif; font-size: 12pt;"><span style="color: blue;">Facebook </span></a><span style="font-family: 'Times New Roman', serif; font-size: 12pt;">and</span><a href="https://plus.google.com/up/start" style="font-family: 'Times New Roman', serif; font-size: 12pt;"><span style="color: blue;"> Google +</span></a><span style="font-family: 'Times New Roman', serif; font-size: 12pt;"> - public social networking tools for keeping tabs on your friends around the world.</span></li>
<li> <span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-language: AR-SA; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: JA;"><a href="http://www.socialcast.com/"><span style="color: blue;">Socialcast</span></a>, <a href="http://www.clearvale.com/mkt/en/"><span style="color: blue;">Clearvale </span></a>, <a href="http://www.jivesoftware.com/social-business/solutions"><span style="color: blue;">Jive </span></a>and <a href="http://www.spigit.com/solutions/products"><span style="color: blue;">Spigit </span></a>- Enterprise social networking tools for keeping abreast of news and status, smoothing workflow, fostering collaboration and spurring idea generation in a corporate context.</span></li>
</ul>One additional consideration is which user base a cloud deployment is intended to serve. This leads to the ideas of: <b>Private, Public and Hybrid Clouds</b><br />
<br />
<ul><li>Private cloud = services and (often hardware) are strictly for a single company's use. Frequently this implies that the company consuming the services will deploy the cloud services on hardware behind their corporate firewall.</li>
<li>Public cloud = services are available for all users anywhere. Though users may need to pay for services... This is the context in which most people experience the cloud because of the heavy reliance start-ups offering SaaS products have on public cloud services.</li>
<li>Hybrid cloud = multiple cloud systems connected in such a way as to allow programs and data to be easily moved between private to public clouds.</li>
</ul>Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-52643504068691519812012-01-31T23:59:00.000-08:002012-02-04T23:03:42.759-08:00Cloud Computing, Virtualization, IaaS, PaaS and SaaS - Part 1<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4-Je3deS0KRu_Hw7ZPhyphenhyphenLZySCS3Cfu2Bp0H9UvrYsVjKCSDCRE5r36KizZ6qMbEoqH27S5ErAvnIed6NBVPCl9w_b0hMbJx7oEHb24cDyy5ds-PDrCbKDx0IXKaqf0grlHFLrX6wnMRjc/s1600/cloud-hosting.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4-Je3deS0KRu_Hw7ZPhyphenhyphenLZySCS3Cfu2Bp0H9UvrYsVjKCSDCRE5r36KizZ6qMbEoqH27S5ErAvnIed6NBVPCl9w_b0hMbJx7oEHb24cDyy5ds-PDrCbKDx0IXKaqf0grlHFLrX6wnMRjc/s320/cloud-hosting.jpg" width="202" /></a></div>What is cloud computing and why is it important?<br />
<br />
On the question of importance, cloud computing addresses several business problems.<br />
<br />
Perhaps the most mundane problem that cloud computing addresses is one of cost for getting applications up and keeping them running.<br />
<br />
<ul><li>Availability </li>
<ul><li>Because of the underlying technology used in cloud computing (Virtualization - more on this next post.) it becomes relatively easy to implement failover and high availability for servers. This makes the customer experience better (fewer interruptions) and allows more routes to addressing business critical applications (lowers barrier to entry on some enterprise applications).</li>
</ul><li>Application deployment & maintenance </li>
<ul><li>Depending on the level of the cloud you engage at (Infrastructure, Platform or Software - more on this next post), the level of IT expertise required to use the applications / services you want can be greatly reduced. Software patching, upgrades, hardware provisioning and maintenance is left to the cloud providers who have this expertise on staff.</li>
</ul></ul><div>These "mundane" aspects of cloud computing enable significantly less mundane capabilities. The most immediate of which are the new business models that become feasible.</div><div><ul><li>Software start ups galore</li>
<ul><li>Because compute power can be purchased on-demand and in relatively small units, capital costs for starting a software company are significantly reduced. This enables more players to try more things in search of the next big thing.</li>
</ul><li>F2P (Free to Play), Freemiums and ad-driven businesses</li>
<ul><li>Because the marginal cost of adding users to a centralized s/w application can be nearly zero when deployed in a dynamically scalable infrastructure such as the cloud provides, companies can explore business models that "give away" software, or more precisely: give away <i>access </i>to software, while closely tracking and tailoring the user experience more easily. This leads to businesses that thrive on micro transactions within the software or on ad revenue instead of from sales of the software itself.</li>
</ul><li>On-line special events</li>
<ul><li>Relatively low cost and rapid provisioning of compute, network and storage resources allows for companies to generate increased community engagement or media attention by using this burst capacity to host periodic special events. For a small fraction of the cost of buying and setting up the h/w and software required to gain this capacity, the same buzz generating potential can be realized.</li>
</ul><li>Mobile</li>
<ul><li>By using the relatively limited compute power of networked mobile devices to drive the user interface and pushing the computationally intensive tasks to servers in the cloud, the capabilities of mobile devices become nearly unlimited. This creates a new class of applications (e.g. voice recognition, navigation, etc) for entrepreneurs to explore and market, driving sales of mobile computing hardware (e.g. smart phones and tablets) or of the applications (as apps or as services) themselves. </li>
</ul><li>Distributed work force, virtual desktops and collaboration</li>
<ul><li>Because public cloud resources can be accessed from any internet connected device and data storage is easily centralized, the cloud makes it more feasible than ever to provision specialized applications to anyone, anywhere, to keep company data centralized and provide tools by which physically separated groups can easily interact and exchange information in real time in order to improve productivity.</li>
</ul></ul></div>As Corporate Social Responsibility (CSR) and sustainability concerns increase, the "green" aspects of cloud computing also become more important.<br />
<br />
<ul><li>Economies of scale in "green" data centers may mean that renting capacity via "the cloud" is greener than you can afford on your own. </li>
<ul><li>Making an efficient data center requires high up front capital and expertise costs. Considerations like the ones below are more affordable by large businesses or by dedicated cloud hosting companies where the risk associated with the capital costs are more acceptable for the long term return.</li>
<ul><li>efficient server h/w</li>
<li>low energy and passive cooling strategies</li>
<li>occupancy based lighting, energy efficient lighting fixtures and daylighting strategies</li>
<li>green power purchase agreements</li>
<li>on-site generation via renewables (e.g. co-gen, bio-fuel based fuel cells, PV, Solar thermal, etc.)</li>
</ul></ul></ul><br />
<a href="http://cursoryknowledge.blogspot.com/2012/02/cloud-computing-virtualization-iaas.html">Next time</a> some details about the the "what" part of the question.Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-72265994732066044932012-01-03T13:23:00.000-08:002012-01-03T13:23:02.836-08:00B-Corporations are Real in California<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2pVK9gKTYFEGQix7HJtrpX_HgZRCC6cEEBlOcd5oh8iY-4rkvEgAIGsi7BFFj3kSlHCL7iZfFVzh5MlZSOUp-yzwEiF8zx2krvgqCw5ItdJ7Uid1b93ajl53wCGgjyFGtd7xoCxkWSGoD/s1600/20120103-patagonia-store-taipei.jpg.492x0_q85_crop-smart.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2pVK9gKTYFEGQix7HJtrpX_HgZRCC6cEEBlOcd5oh8iY-4rkvEgAIGsi7BFFj3kSlHCL7iZfFVzh5MlZSOUp-yzwEiF8zx2krvgqCw5ItdJ7Uid1b93ajl53wCGgjyFGtd7xoCxkWSGoD/s320/20120103-patagonia-store-taipei.jpg.492x0_q85_crop-smart.jpg" width="320" /></a></div>It looks like the <a href="http://www.leginfo.ca.gov/pub/11-12/bill/sen/sb_0201-0250/sb_201_bill_20110208_introduced.pdf">legislation making B-Corporations legal</a> in California has passed and <a href="http://cursoryknowledge.blogspot.com/2011/02/flexible-purpose-corporations.html">one big name corporation has signed up</a>.<br />
<br />
Read <a href="http://cursoryknowledge.blogspot.com/2011/02/flexible-purpose-corporations.html">this previous post</a> for more thoughts on what this means.<br />
<br />
It's a good step towards making it legal, by changing the focus on <a href="http://cursoryknowledge.blogspot.com/2010/03/to-be-good-member-of-society-you-must.html">shareholder primacy</a>, for a public corporation to even consider doing what you or I would see as mandatory in our dealings with other people and with our communities.Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-1972237364532550712012-01-03T12:58:00.000-08:002012-01-03T13:08:25.073-08:00More on Virtual Daylighting<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_CP4NzsdjNq4Q94VPskCLm5c_9ggWu0YeBGj_4H74cxu_y9AD_mdjzZm0rOiTeCpxQ0ZEmnir8QTKb8V-1ZEnFXNgBBC431O5rCU_Hlx9pIoS2YEMcypR8PJE_4w3aYamdI3uG55hQ2RZ/s1600/skylight-clouds.jpg.492x0_q85_crop-smart.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="155" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_CP4NzsdjNq4Q94VPskCLm5c_9ggWu0YeBGj_4H74cxu_y9AD_mdjzZm0rOiTeCpxQ0ZEmnir8QTKb8V-1ZEnFXNgBBC431O5rCU_Hlx9pIoS2YEMcypR8PJE_4w3aYamdI3uG55hQ2RZ/s320/skylight-clouds.jpg.492x0_q85_crop-smart.jpg" width="320" /></a></div>I posted earlier about an experiment with <a href="http://cursoryknowledge.blogspot.com/2010/05/experiment-with-virtual-daylighting.html">virtual daylighting</a> on a small scale using indirect light.<br />
<br />
<a href="http://www.treehugger.com/gadgets/office-ceilings-mimic-lighting-of-cloudy-days-with-new-led-design.html">Here is a post</a> about a direct experiment where the designer replicates the sky, clouds and all, using arrays of LEDs.<br />
<br />
Probably not an energy efficient use of LEDs but an interesting experiment that requires some thought about the trade-offs between energy efficiency and productivity gains.Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-46745258527249876112011-02-14T23:14:00.000-08:002011-02-15T08:12:25.047-08:00Flexible Purpose Corporations - California making it legal for corporations to "care" about something other than money<div class="separator" style="clear: both; text-align: center;"><a href="http://www.greenbiz.com/sites/default/files/imagecache/wide_large/3leggedstool.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="http://www.greenbiz.com/sites/default/files/imagecache/wide_large/3leggedstool.jpg" /></a></div>In a <a href="http://cursoryknowledge.blogspot.com/2010/03/to-be-good-member-of-society-you-must.html">previous post</a>, I wrote about shareholder primacy (the law governing corporations actually <b>requires</b> the CEO to pursue shareholder profit even at the expense of other corporate goals or values) and how that makes it very difficult for a corporation to pursue goals that are not directly tied to increasing bottom line profits.<br />
<br />
According to this <a href="http://www.greenbiz.com/blog/2011/02/14/california%E2%80%99s-move-legalize-sustainable-business">Green Biz blog</a>, California is getting ready to pass a law (<a href="http://www.leginfo.ca.gov/pub/11-12/bill/sen/sb_0201-0250/sb_201_bill_20110208_introduced.pdf">2011 SB201</a>) which <b>makes it legal</b> for corporations established in California "...<b>to include a social and environmental mission that is given equal weight, perhaps even greater weight, than profits</b>..." It protects CEOs and boards of directors in pursuit of such goals and it provides shareholder mechanisms to enforce the pursuit of the same.<br />
<br />
That's an improvement.<br />
If you can get companies to stop <a href="http://www.delawareintercorp.com/t-whyincorporateindelaware.aspx">incorporating in Delaware</a>.<br />
By which I mean that "traditional" motives, training and culture need to change in addition to the law. <a href="http://www.amazon.com/Cod-Biography-Fish-Changed-World/dp/0140275010">Profit is so deeply rooted in American culture and history</a> that it is difficult to even see <a href="http://cursoryknowledge.blogspot.com/2010/05/in-thrall-of-old-ideas.html">how enthralled by the old ideas</a> we are.Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0tag:blogger.com,1999:blog-5328915521089085385.post-44090954945265744382010-12-31T23:38:00.000-08:002010-12-31T23:42:30.868-08:00Saving the Savages = Breaking the System<div class="separator" style="clear: both; text-align: center;"><a href="http://ecx.images-amazon.com/images/I/41f4u54o23L._SL500_AA300_.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="200" src="http://ecx.images-amazon.com/images/I/41f4u54o23L._SL500_AA300_.jpg" width="200" /></a></div>I just finished reading <a href="http://www.amazon.com/Wayfinders-Ancient-Wisdom-Matters-Lecture/dp/0887847668/ref=sr_1_1?s=books&ie=UTF8&qid=1293865445&sr=1-1"><i>The Wayfinders - Why Ancient Wisdom Matters in the Modern World</i></a> by Wade Davis. Besides being one of the best arguments I've read on why sustainability matters and where we might "relearn" how to achieve it, there was a section on the consequences, however well intended, of misunderstanding the context of a culture and of applying the wrong solution to a problem because of it.<br />
<br />
The story is of the people of the Kasuit desert in northern Kenya. For hundreds of years, they lived in this desert. They developed a culture that embraced the difficulties of living in the desert and, therefore, did just fine. Until severe drought, combined with nearby wars brought this region to the attention of the West. At which point, the United Nations took actions to "save" them from themselves.<br />
Following is my summary of Wade's story.<br />
<br />
The culture the originally developed worked like this:<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMqGvnWQSmJTemNMapw1-rBWkChnZQBUdRYGkqiL3NWZ7UrUQNgdrq4fWz5aYlWvBpMFmjfgcb4pmfwMLIrDlEu-zqXjeekqMJEguFMDOKmpUJIjYQ2wGUNmDrrjHxsQRQSnNWZttl-_Px/s1600/culture%253Dsurvival%2528working%2529.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="298" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMqGvnWQSmJTemNMapw1-rBWkChnZQBUdRYGkqiL3NWZ7UrUQNgdrq4fWz5aYlWvBpMFmjfgcb4pmfwMLIrDlEu-zqXjeekqMJEguFMDOKmpUJIjYQ2wGUNmDrrjHxsQRQSnNWZttl-_Px/s640/culture%253Dsurvival%2528working%2529.JPG" width="640" /> </a><br />
To ensure that enough livestock survived severe drought, large herds of livestock were required.<br />
To maintain large herds required large families, stewardship of the land and a nomadic lifestyle (so that the carrying capacity of any one area was not exceeded).<br />
The requirement to have large families resulted in polygamy being common. But to maintain social harmony among the young men in a group where there were not enough women for every man, the culture developed rites around sending young men away to tend the herds. These rites bound together the men involved in them, creating life-long social bonds which reinforced the "commonness" of their use of the land (avoiding the "tragedy of the commons"). These rites also instilled a large amount of prestige in the men, allowing them to accept the situation which supported the large families.<br />
<br />
Then the UN intervened and added a few things:<br />
<div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6sphRu_3EUhS4bpjuJT7X0b0TlHdlp5EBq1bTr72QlLA0IhpYa03Fbq-u9n2-nDEe_oDcmnNPpl4cq4FTQoY8MWU3TNKHByN-hQ23-mGjLeMAt9jDr9j07qgyXynz48aKY02xYhp2CvtG/s1600/culture%253Dsurvival%2528broken%2529.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6sphRu_3EUhS4bpjuJT7X0b0TlHdlp5EBq1bTr72QlLA0IhpYa03Fbq-u9n2-nDEe_oDcmnNPpl4cq4FTQoY8MWU3TNKHByN-hQ23-mGjLeMAt9jDr9j07qgyXynz48aKY02xYhp2CvtG/s640/culture%253Dsurvival%2528broken%2529.JPG" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiE2ZaP7zo6X1wDpVJCLsnnkGSuS1G69FoooKI84W6xdguh4joBxD8NLqwf7MnIev5o2_dxN42I3xUbbkcsCk4GAkoCFofcR5PJmpoXk7QPGzX5vBJSzO_7aY7laQrVtCstzwB89mgSWJkB/s1600/culture%253Dsurvival%2528broken%2529.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><br />
</a></div>First they added the idea of private ownership of the land and, thereby, permanent settlement. This required the reduction of herd size which quickly led to dependence of outside aid to keep the people alive.<br />
Outside aid included western education. The education broke the traditional rites of passage (by instilling the idea that such a life was "beneath" an educated man), which broke the personal ties which were necessary, among the other factors, to allow life in the harsh environment.<br />
<br />
The entire culture collapsed and, along with it, the ability to sustain life in that place.<br />
All with the best intentions...<br />
...but without sufficient respect for the wisdom of the people who had learned to live within the limits of the place they inhabited.Anonymoushttp://www.blogger.com/profile/03916477205554278144noreply@blogger.com0