hybird cloud

connected with public cloud and containerization, such as Docker, Kubernets

wiki :

official website :





讀 //

以Dashboard 或Oenstack API 管理.

Openstack 已形成生態系,

The OpenStack foudation //

我從頭到尾看了 一遍, 沒有一家台灣公司. 這… 一定要有自覺. 落後到連車尾燈看不到, 跟上去

讀 :

  • OpenStack-Ansible Deployment Guide //這要讀 Ansible 之前寫過


    OpenStack ansible 簡介:

    Ansible, LXC (Linux container), 這些觀念都是環環相扣 

  • 部署

  • Deployment guide //

  • 安裝

  • Ocata Installation Tutorials and Guides //
  • OpenStack Documentation //

Architecture 有 3個:







Google, AI, Machine Learning

Google cloud platform (note5)

Target : the enterprise cloud market

  • Position- To be a developer-friendly platform
  • Weakness

1) Not strong and no impression  on cloud service and the enterprise segment

2) No contribution to the open source community before.


  • Stength : ASIC, GPU and TPU hardware in its cloud
  • Opportunity

1) begin to work with open source projects (note3)

  • Cloud Native Computing Foundation- the open-source container management tool

run by the Linux Foundation (note4)

Other partners in the new foundation include AT&T, Box, Cisco, Cloud Foundry Foundation, CoreOS, Cycle Computing, Docker, eBay, Goldman Sachs, Huawei, IBM, Intel, Joyent, Kismatic, Mesosphere, Red Hat, Switch SUPERNAP, Twitter, Univa, VMware and Weaveworks.

Absent : Microsoft, Amazon, Pivotal, and  Taiwanese tech companies)

the popular open source container orchestration system

  • TensorFlow for machine learning

-Spanner for launching massive distributed databases

-Draco for 3D graphics compression

2) To be a  developer-friendly platform


1)letting customers run whatever open source stack they choose on Google’s infrastructure,

2)releasing and supporting open source projects and making the ecosystem

3)the partners who build tools and technologies on top of GCP, a first class citizen on the platform.

4) treating them as part of the whole and the net is bringing the tech you want and using Google technology or using any of the [partner] services

The KSF :

1)Being open to win the mind shares of developers

2) much more supportive of the open source community makes people feel better about Google and makes developers feel better about working with their tools because they can avoid lock-in

  • Threat : peers: AWS (2006. 1st public cloud, market leader, 1st mover), Microsoft, IBM
  • Strategy

UsingKubernetes, the popular open source container orchestration system offer robust open source tools, something that surprised some people in this market.

  • 4 ways Google will enable enterprises to adopt machine learning and AI (note2)

1). Machine learning computing in Google Cloud

a deep learning algorithm can have tens of millions of parameters, training these machine learning models requires enormous computational resource

the Cloud Machine Learning Engine. This capability is designed for companies with data scientists and machine learning experts who are able to build their own unique machine learning models with libraries such as Tensorflow.

Google’s infrastructure as the solution to speed training times and improve the return on investment. Google has specialized ASIC, GPU and TPUhardware in its cloud to accelerate training and improve the ROI with on-demand cloud resource utilization. After the model is trained, it is deployed in range of platforms—from on-premise to mobile devices.

2. Algorithms and pretrained machine learning models

建ML model 需用 the machine learning engine, 用 Google’s pre-trained models (full list) using APIs to add machine learning capability to their applications, such as understanding natural language, images and natural language.

An API beta for understanding videos

demo: This 3-minute video of the demonstration of the Cloud Video Intelligence beta

3. Google acquires Kaggle for data

Google acquired Kaggle for data sets and talent. Kaggle, founded in 2010, is a community of 850,000 data scientists from around the world that hosts competitions to create the most accurate predictive models and market models, as well as to acquire new public data sets in a variety of fields.

4. Expertise

the Advanced Solutions Lab for customers with ambitious goals to develop machine learning to solve complex problems.




(note4: The mission of Linux foundation :The mission of this new foundation is to “help facilitate collaboration among developers and operators on common technologies for deploying cloud native applications and services,” )


(Reference : Google Cloud Platform 入門)



Apache Flink

這是個計算引擎, 號稱" 4G of Big Data" (note1), 快, 易用,開源, 效能佳, 但沒有儲存系統

  • Batch Processing
  • Interactive processing
  • Real-time stream processing
  • Graph Processing
  • Iterative Processing
  • In-memory processing

Flink is an alternative of Mapreduce, it processes data more than 100 times faster than MapReduce.

Flink is independant from hadoop but it can use hdfs to read, write, store, process the data. Flink does not provide its own data storage takes data from distributed storage.

Flink  ecosystem:   (note2)


 Storage: 讀寫別家的資料庫大概都沒什麼問題

  • HDFS – Hadoop Distributed File System
  • Local-FS – Local File System
  • S3 – Simple Storage Service from Amazon
  • HBase – NoSQL Database in Hadoop ecosystem
  • MongoDB – NoSQL Database
  • RBDBMs – Any relational database
  • Kafka – Distributed messaging Queue
  • RabbitMQ – Messaging Queue
  • Flume – Data Collection and Aggregation Tool


Deploy: 能分配部署資源 :

  • Local mode – On single node, in single JVM
  • Cluster – On multi-node cluster, with following resource manager
    • Standalone – This is the default resource manager which is shipped with Flink
    • YARN – This is very popular resource manager, it is part of Hadoop, introduced in Hadoop 2.x
    • Mesos – This is a generalized resource manager.
  • Cloud – on Amazon or Google cloud

Runtime :

the Distributed Streaming Dataflow, which is also called as kernel of Apache Flink. This is the core layer of flink which provides distributed processing, fault tolerance, reliability, native iterative processing capability, etc.





  • Streaming – Flink is a true stream processing engine.
  • High performance – Flink’s data streaming Runtime provides very high throughput
  • Low latency – Flink can process the data in sub-second range without any delay
  • Event Time and Out-of-Order Events – Flink supports stream processing and windowing where events arrive delayed or out of order
  • Lightning fast speed – Flink processes data at lightning fast speed (hence also called as 4G of Big Data)
  • Fault Tolerance – Failure of hardware, node, software or a process doesn’t affect the cluster
  • Memory management – Flink works in managed memory and never get out of memory exception
  • Broad integration – Flink can be integrated with various storage system to process their data, it can be deployed with various resource management tools. It can also be integrated with several BI tools for reporting
  • Stream processing – Flink is a true streaming engine, can process live streams in sub-second interval
  • Program optimizer – Flink is shipped with an optimizer, before execution of a program it is optimized
  • Scalable – Flink is highly scalable. With increasing requirements we can scale flink cluster
  • Rich set of operators – Flink has lots of pre-defined operators to process the data. All the common operations can be done using these operators
  • Exactly-once Semantics – It can maintain custom state during computation
  • Highly flexible Streaming Windows – In flink we can customize windows by triggering conditions flexibly, to get required streaming patterns. We can create window according to time t1 to t5 and data driven windows.
  • Continuous streaming model with backpressure – Data streaming applications are executed with continuous (long lived) operators. Flink’s streaming engine naturally handles backpressure.
  • One Runtime for Streaming and Batch Processing – Batch processing and data streaming both have common runtime in flink
  • Easy and understandable Programmable APIs – Flink’s APIs are developed in a way to cover all the common operations, so programmers can use it efficiently.
  • Little tuning required – Requires no memory, network, serializer to configure

初看這Apache Flink, 電視台轉型需用到,以往直播用SNG車, 上衛星, 現在改串流技術,  光這樣成本就不知省多少,用途滿廣, 也可處理髒資料,推薦產品用, 作預測.










  1.  入門:
  • 讀這份 document :

這software engineering 必用

2. Typical Docker Platform Workflow  有5個步驟 :

  1. Get your code and its dependencies into Docker containers:
  2. Configure networking and storage for your solution, if needed.
  3. Upload builds to a registry (ours, yours, or your cloud provider’s), to collaborate with your team.
  4. If you’re gonna need to scale your solution across multiple hosts (VMs or physical machines), plan for how you’ll set up your Swarm cluster and scale it to meet demand.
  5. Finally, deploy to your preferred cloud provider (or, for redundancy, multiple cloud providers) with Docker Cloud. Or, use Docker Datacenter, and deploy to your own on-premise hardware
  1. Docker component  for Windows //


  • Docker for Windows requires 64bit Windows 10 Pro, Enterprise and Education (1511 November update, Build 10586 or later) and Microsoft Hyper-V. Please see What to know before you install for a full list of prerequisites.
  • Docker for Windows requires Microsoft Hyper-V to run. After Hyper-V is enabled


中文的Docker介紹: Docker OpenSource專案簡介








  • for IaaS

OpenStack是美國國家航空暨太空總署Rackspace共同打造的雲端開源軟體,以Apache許可證授權,並且是一個自由軟體和開放原始碼項目,來打造基礎設施即服務(Infrastructure as a Service) (note2)

  • 3個 大modules:

運算模組網通模組儲存模組,加上一套集中式管理的儀表板模組,來組合成一套OpenStack共享服務,並且以提供虛擬機方式,對外帶來運算資源,以便利彈性擴充或調度 (note1))

所以網管用的, 網通硬體, 程式化,虛擬化是必然. 硬體的軟體化, 虛擬化.ek4

各模組(套件), 請看 (note1)


  • 網通模組(套件,module) :Neutron



類似 Amazon AWS 的 VPC。



  • Nova運算專案[1]
  • Swift物件導向數據存貯專案[2]
  • Glance虛擬機器磁碟映像檔(Virtual Machine Image)傳送服務[3] [4]
  • Horizon- 提供簡易Web界面和管理控制台[5]
  • Cinder – 提供Block資料存取
  • Keystone – 提供身份驗證機制
  • Neutron – 提供網路管理功能
  • Trove – 提供資料庫管理功能
  • Sahara – 提供海量資料運算佈署功能
  • Ceilometer – 提供計量與監控功能
  • Heat – 提供自動延展虛擬機功能


  • Trove 資料庫服務套件 (Database as a Service)

Trove主要負責銜接簡化實際資料庫的使用,提供OpenStack各個服務一個具延展性且可靠的雲端資料庫服務(Cloud Database-as-a-Service),Database服務包含了銜接傳統關聯式資料庫與新興非關聯式資料庫.

  用 Ubuntu os .

這是好東西, 對的方向, 省錢, 加值. 亞洲接受度滿快的.





(OpenStack 資源整理:


IaaS 看這幾家, 夠了,

我以前跟很多IDC 打交道,  菲律賓, 新加坡,  印尼, 日本, 泰國, 越南,馬來西亞…等

大陸的IaaS 要了解一下

  1. AWS
  2. AT&T
  3. GoGrid //  (note1)

GoGrid is a cloud infrastructure service, hosting Linux and Windows virtual machines managed by a multi-server control panel and a RESTful API.

1.RackSpace // (note2) (note6)



2.世紀互聯 // www.  (note3))

3. 華勝天成 // IaaS管理系統 (note4)




4.神州數碼 // 營運管理平台  (note5)






(note6: Rackspace company profile)

  • Joe

  • 分類

  • 標籤

  • 文章存檔

  • Follow Joe on
  • 輸入電子郵件地址以關注此網誌,並透過電子郵件接收新文章通知。

  • 廣告
%d 位部落客按了讚: