Translate

Tuesday, June 3, 2014

Andy Hunt Video Talk Pragmatic Thinking by Ismael Marin

Video talk at the Universidad Iberoamericana Leon in Mexico at the CESLG 08, in which andy talks about his career and his new book Pragmatic Thinking and Learning, he also gives tips to be a better programmer. http://ift.tt/1mOObX5

Thursday, May 22, 2014

Dave Thomas - RubyConf AU 2013 Closing Keynote by Ruby Australia

Dave Thomas needs no introduction to Ruby programmers. As co-author of "Programming Ruby: The Pragmatic Programmers' Guide" - fondly known as the "Pickaxe", Dave was instrumental in spreading Ruby beyond its birthplace in Japan.

Monday, May 19, 2014

What is Fog Computing?


Are you in the mood for a new technology buzzword?  CISCO's marketing department is rolling with the term 'Fog Computing'.
The Fog is an extension of the Cloud.  So in Fog Computing, data is stored closer to the devices we use to access the internet. The benefit is a reduction of bandwidth and latency.
CISCO routers already live in this grey area between out devices and big data providers. CISCO is adding Linux operating systems to some of their routers, and manage frequently accessed data with a distributed network of these router on the edge of the cloud.
by David Flectcher

Naturally several definitions of Fog Computing mention another buzzword, The Internet of Things. But the concept of caching data nearby isn't new.   My paper Smart Technology for Big Data talks about how Facebook caches images close to users based on how likely the images will be accessed.  Today's article Forget 'the Cloud'; 'the Fog' Is Tech's Future in the Wall Street Journal reluctantly embraces the new term.
Welcome to the Fog.

Monday, May 5, 2014

An IT Department at your fingertips : Amazon's EC2

Amazon Web Services is an essential tool for a versatile software consultant. It gives us a vast array of possible configurations. This comes in handy when working out new ideas. But it's also great for installing and testing legacy code.

EC2 - Virtual Servers in the Cloud




The EC2 tab lets us create virtual machines of many flavors. You can test out AWS for a year using the free usage tier.

Don't have a copy of windows handy? Spin up an EC2 instance with Windows, then connect using remote desktop software.  You can also use a Linux box any time you desire.

There's much that can be done with these instances, even install X-Windows on a linux instance and connect from a Chromebook using Chrome Remote Desktop.


Tuesday, January 7, 2014

Big Data Analytics by ICGX

The data revolution has only just begun. Everyone is talking about Big Data. Big Data grows up - Forbes Business opportunities is Big Data - INC. Big Data powers evolution decision making - WSJ How Big Data got so big - NYT Big Data is hot? Now what? - Forbes Businesses "freak out" over Big Data - Information Week 2012: The year of Big Data - WSJ The age of Big Data - NYT But it's not just hype. The world's data is doubling every 1.2 years. There are 7 billion people in the world. 5.1 billion of them owns cell phone. Each day, we send over 11 billion texts, watch over 2.8 billion YouTube videos and preform almost 5 billion google searches. And we're not just consuming it. We're creating it. We are data agents. We generate over 2.4 quintillion bytes everyday from consumer transactions, communication devices, online behavior, streaming service. In 2012, the world’s information totaled over 2 zetabyes. That’s 2 trillion gigabytes. By 2020, that number will be 35 trillion. We will need 10x more servers, 50x more data management, 75x more files to handle it all. If you're like most companies, you aren't ready. 80% of this new data is unstructured. It is too large, too complex, and too disorganized to be analyzed by traditional tools. There are 500K computer scientists yet only 30K mathematicians. We will fall short of the talent need to understand Big Data by at least 100K. To find opportunities in Big Data, we need new tools and new talent to mine this information and find value. We need Big Data Analytics. Big Data Analytics is more than technology. It’s a new way of thinking. It will help companies better understand customers, find hidden opportunities, even help our government better serve citizens and mitigate fraud. It will inspire hundreds, thousands and even millions of new startups. It will alter the landscape across virtually every industry and finally answer the questions looming over every CEO's head, "How can my business use Big Data?", "What problems can it solve?", "Who should be leading the charge, CIO, CMO, or Chief Data Scientists ?". In every revolution, there are opportunities that will be seized only by those armed with the right tools and right strategy. We are at the beginning of the Big Data Revolution.

Thursday, January 2, 2014

Running OpenTSDB on Amazon EC2

Although there are cheaper alternatives for production systems, It's easy enough to get The Open Time Series Database OpenTSDB running on an EC2 instance of Amazon Web Services.

  1. First you'll need to run HBase on EC2
  2. Make a data directory mkdir hbase_data
  3. vi hbase-0.94.13/conf/ hbase-site.xml
  4. Using vi update the hbase.rootdir property value to: file:///home/ec2-user/hbase-0.94.13/hbase-\${user.name}/hbase
  5. sudo yum install git
  6. git clone git://github.com/OpenTSDB/opentsdb.git
  7. sudo yum install automake
  8. yum install gnuplot
  9. cd opentsdb
  10. ./build.sh
  11. env COMPRESSION=NONE HBASE_HOME=path/to/hbase-0.94.X ./src/create_table.sh
  12. tsdtmp=${TMPDIR-'/tmp'}/tsd
  13. mkdir -p "$tsdtmp" 
  14. ./build/tsdb tsd --port=4242 --staticroot=build/staticroot --cachedir="$tsdtmp"
  15. In AWS, click on your EC2 instance, then click "Security Groups" at the bottom left.  Click on the default group, then click the "inbound" tab.  You can now open the ec2 port 4242. 
Your ip address on port 4242 will display the web UI for your instance of OpenTSDB:








  • Thursday, December 26, 2013

    Running HBase on Amazon EC2

    1. Create an Amazon Linux EC2 instance. 
    2. Log into your EC2 Instance using ssh.
    3. sudo yum install java-1.6.0-openjdk
    4. wget http://www.apache.org/dist/hbase/hbase-0.94.13/hbase-0.94.13.tar.gz
    5. tar xfz hbase-*
    6. vi .bashrc
    7. Add this line at the bottom of the file JAVA_HOME=/usr/java/default
    8. sudo vi /etc/hosts
    9. Comment out the localhost line: #127.0.0.1   localhost localhost.localdomain
    10. cd  hbase-*
    11. Start HBase ./bin/start-hbase.sh
    12. Check log files cat logs/hbase-*

    Tuesday, December 24, 2013

    The Journal of Trading: Smart Technology for Big Data


    Smart Technology for Big Data was published in the Winter edition of Journal of Trading.  You need to register to read them.  Here's the Abstract:

    This article provides an underlying structure for managing the big data phenomenon. Innovations and tools fundamental to handling big data are highlighted, and we look at how these technologies are being implemented in the financial industry. 

    See more at: http://www.iijournals.com/doi/abs/10.3905/jot.2013.9.1.057

    Wednesday, December 18, 2013

    Institutional Investor Journals: Big Data Article


    UPDATE:   Smart Technology for Big Data was published in the Winter edition of Journal of Trading, so the links below no longer work. You can access the article here: Smart Technology for Big Data (You'll still need to register if you haven't)




    My article Smart Technology for Big Data is published under advanced content at the Institutional Investor site.  You'll need to complete the free registration to read it. Enjoy!


    Monday, December 9, 2013

    Grace Hopper


    Today's Google doodle is for Grace Hopper, a computer programmer who would have been 107 today.


    Grace was an admiral in the US Navy and worked on one of the first computers in 1945. She worked on the committee that created the COBOL programming language, and has a US Navy destroyer, the USS Hopper named after her.


    Grace's intelligence and charm shines through in this interview on David Letterman in 1986.  In a field where women are still underrepresented, it's tremendous that there are women like Grace Harper in computer science whom we cal all revere.