OpenShift

https://technews.tw/2019/03/18/hybridcloud-redhat-openshift/

hybird cloud

connected with public cloud and containerization, such as Docker, Kubernets

wiki : https://en.wikipedia.org/wiki/OpenShift

official website :https://www.openshift.com/

 

廣告

Fedora

official site : https://getfedora.org/

pc version for download : https://getfedora.org/zh_TW/workstation/

wiki: https://zh.wikipedia.org/wiki/Fedora

Fedora專案以不同方式發行Fedora[9]

  • Fedora DVD/CD——包含了所有主要軟體包的DVD或CD套裝;
  • Live鏡像——CD或DVD大小的光碟鏡像,可用於建立Live CD或從USB裝置啟動,並可選安裝到硬碟;
  • 最小CD——用於通過HTTPFTPNFS安裝。[10]

Live 鏡像 :

Live USB 是USB隨身碟或USB硬碟,裡面含有完整的作業系統,可以被用來開機(booting)。

Live USB很像live CD,但基本上有能力更改設定,而且可以把軟體安裝回USB裝置上。

像live CD一樣,live USB可以使用在嵌入式系統系統管理(embedded systems system administration),資料還原(data recovery),或是不需要把作業系統安裝到主機硬碟(local hard disk drive)裡的測試。許多作業系統,包含微軟的Windows XP Embedded和許多Linux套件BSD可以被安裝在隨身碟上使用。

您可以通過Fedora Live USB Creator[11]或UNetbootin建立Live USB版本的Fedora。

(11: https://fedoraproject.org/wiki/Infrastructure/Fedorahosted-retirement // live usb creator 看來是倒了)

system requirement :

  • CD或DVD驅動器。
  • 1GHz處理器或更快
  • 推薦1GB的記憶體及以上(RAM)
  • 推薦10GB的永久儲存空間(硬碟機)

如果您的電腦沒有CD或DVD驅動器,或者是不能夠從該驅動器啟動,那麼你可能需要從USB儲存裝置,如USB快閃記憶體驅動器來安裝Fedora。

仍被支援的版本 :  28.29

最新的Fedora版本為29,發布於2018年10月30日[25]

Fedora 29的​主​要​特​性​如​下​:

  1. 整合了​GNOME桌​面​最​新​版​本​ 3.30。

GNOME : https://zh.wikipedia.org/wiki/GNOME

一個完全由自由軟體組成的桌面環境。它的目標作業系統Linux,但是大部分的BSD系統亦支援GNOME。

 

 

 

 

3 best practices for bootstrapping an open source business

前幾天才寫創業要從自己開始,但今天就看到一篇文章提到做一個開源的軟體生意不要自獨做. 顯然很多事沒標準答案.  要看情況

關於創業很多都是聚焦在籌資, 怎麼搞錢, 如何如何.

我反而觀察現在因為技術的普及,創業的成本已大幅降低. 不應該聚焦過多在資金面.  open souce  相關的項目, 成本已經很低,  相對ROI應該極高.

但 That open source startups are hard to find in the investment-first ecosystem is not surprising, because they’re usually started by people who actually build the product.

Most of the time, seeking early stage investment for an open source product doesn’t make financial sense.

On the other hand, there’s much to be gained from the business and marketing knowledge in local startup communities, so being sequestered from them can put open source developers at a disadvantage.

很多公司的觀念也是這樣,  我有親身體驗.  過度強調做產品, 而沒有整體商業概念, 沒有財務觀念, 沒有市場知識, 不是菜市場歐,是產品的市場, 是全球各區各國的市場. 有些也是老闆或主管的私心造成的, 台灣企業很嚴重. 真的.

若是開一家 Open source 的公司, 竟然建議不要獨自創業, 這引起我的好奇.

  • Don’t do this on your own

Take a look at most developers’ personal projects and you’ll find that they’re usually very personal. We have a tendency to base our side work on the one thing we can’t base our daily work on, and often this is why so many open source side projects quickly fade into obscurity. The developer focuses so much energy on writing “elegant" software that they forget to release usable software.  We can be so bent on our own use-case that we forget to build something for a realistic user base.

That’s why having at least one developer partner when building an open source product is so important.

Having someone capable of questioning your decisions and giving new perspectives is invaluable, and it also makes you accountable to someone. Projects with multiple developers have a much higher completion rate because team members tend to keep each other from slacking off.

OK, 我理解了,別太個人化太求完美,至少找一個夥伴. 也比較會怠惰

另外, 提到,

Have a revenue model in mind from the beginning.

Release early, improve regularly

Build your business by thinking like a businessperson

這些細節我就比較清楚了.

Ubuntu

basic info: https://zh.wikipedia.org/wiki/Ubuntu

www.ubuntu.com

“目前Ubuntu共有六個長期支援版本(Long Term Support,LTS):Ubuntu 6.06、8.04、10.04、12.04、14.04、16.04與18.04。自Ubuntu 12.04起,電腦版與伺服器版都有5年支援周期。而之前的長期支援版本為電腦版3年,伺服器版5年。 "

目前Ubuntu正式支援的衍生版本包括:

  • Kubuntu:採用KDE作為預設的桌面環境,以滿足偏愛KDE的Ubuntu用戶。
  • Edubuntu:為教育量身定做,包含很多教育軟體,可以幫助教師方便的搭建網路學習環境,管理電子教室。採用Unity介面。
  • Xubuntu:屬於輕量級發行版,使用Xfce4作為預設的桌面環境。
  • Lubuntu:使用LXDE桌面環境的輕量級發行版[81],從10.04版本開始正式發行。
  • Ubuntu Server Edition:自Ubuntu 5.10版(Breezy Badger)起,與電腦版同步發行,[82]可當作多種軟體伺服器,如電子郵件伺服器、基於LAMP的Web網站伺服器、DNS伺服器、檔案伺服器與資料庫伺服器等。[82]伺服器版通常不預裝任何桌面環境,與電腦版本相比,佔用空間少,執行時對硬體要求較低,最少只需要500MB硬碟空間和64MB記憶體。[31]
  • Ubuntu Studio:適合於音訊,視訊和圖像設計的版本。使用Xfce4作為預設的桌面環境。
  • Mythbuntu:一套基於Ubuntu的面向媒體中心電腦的發行版,Mythbuntu=MythTV+Ubuntu,MythTV是其中關鍵的軟體包,用於實現媒體中心等功能。它沒有包含一些不必要的程式,如OpenOffice, Evolution和完全安裝的Gnome。
  • Ubuntu Kylin(優麒麟):語言的預設設定為簡體中文,為中國用戶專門客製化。[83][84]
  • Ubuntu MATE:針對老舊桌上型、筆記型、樹莓派(Raspberry Pi)電腦,及硬體效能等級不高,或喜歡簡潔、不用特效桌面環境者,使用MATE桌面環境的Ubuntu發行版。
  • Ubuntu for Android:在Android手機上運行的Ubuntu。
  • Ubuntu Touch:基於Ubuntu和Android的手機/平板作業系統。
  • Ubuntu TV:用於智慧型電視的版本。

 

  • 2 GHz dual core processor or better
  • 2 GB system memory
  • 25 GB of free hard drive space
  • Either a DVD drive or a USB port for the installer media
  • Internet access is helpful

Installation guide : https://tutorials.ubuntu.com/tutorial/tutorial-install-ubuntu-desktop?_ga=2.63874671.1166212260.1552626236-567610358.1552626236

 

 

 

 

 

Open science

MaxthonSnap20170415132939.png

 

研究開源化.  這是遲早的事,  越快越好, 過去很多事都很不合理, 這行給我印象不好, 學閥多, 沒風骨開放透明才能正常化.

Open source application for big data

1. Hadoop

  • 這我知 要細看一下網站及文件
  • OS:Windows、Linux 和 OS X
  • website:http://hadoop.apache.org

2. Hypertable

  • Hypertable 在互联网公司当中非常流行,它由谷歌开发,用来提高数据库的可扩展性
  • 与 Hadoop 兼容,提供商业支持和培训。
  • OS:Linux 和 OS X
  • website:http://www.hypertable.com

3. Mesos

  • Apache Mesos 是一种资源抽象工具,有了它,企业就可以鼗整个数据中心当成一个资源池,它在又在运行 Hadoop、Spark 及类似应用程序的公司当中很流行
  • OS:Linux 和 OS X
  • website:http://mesos.apache.org

4. Presto 

  • Presto 由 Facebook 开发,自称是“一款开源分布式 SQL 查询引擎,用于对大大小小(从 GB 级到 PB 级)的数据源运行交互式分析查询
  • OS:Linux
  • website:https://prestodb.io

5. Solr

  • 这种“快若闪电”的企业搜索平台声称高度可靠、扩展和容错
  • OS:与操作系统无关
  • website:http://Lucene.apache.org/solr/

6. Spark

  • 這我寫過
  • Apache Spark 声称,“它在内存中运行程序的速度比 Hadoop MapReduce 最多快 100 倍,在磁盘上快 10 倍
  • OS:Windows、Linux 和 OS X
  • website:http://spark.apache.org

7. Storm

  • Apache Storm 用来处理实时数据
  • OS:Linux
  • 相关网站:https://storm.apache.org

我提過這些技術該如何看,我有興趣的是1.4.5.6,但目前我個人用不到, 要考量時間的機會成本. 不是瞎學就能解決問題, 台灣的企業有多少需要big data 作決策, 擔心市場應用的程度.  我只是要找到應用解決問題,數據要多大,坦白說對我並不重要, 黑貓白貓, 抓的到老鼠就是好貓.

我會繼讀這些文件啦.

Ansible 初探

Ansible (note1)

  • an open-source automation engine that automates software provisioning, configuration management, and application deployment (note2)
  • IT automation engine for using to drive complexity out of their environments and accelerate DevOps initiatives.
  • Ansible, an open source community project sponsored by Red Hat, is the simplest way to automate IT.
  • Ansible is the only automation language that can be used across entire IT teams – from systems and network administrators to developers and managers.
  • Ansible by Red Hat provides enterprise-ready solutions to automate your entire application lifecycle – from servers to clouds to containers and everything in between.
  • Ansible Tower by Red Hat is a commercial offering that helps teams manage complex multi-tier deployments by adding control, knowledge, and delegation to Ansible-powered environments.

The goal of automation process is for update without impact of operational capacity

IT automation, Agile development, DevOps, Deployment, Applicaiton update, Testing,

  • CD,CI (Continus Delivery, Continus Integration)  

CI systems are build systems that watch various source control
repositories for changes, run any applicable tests, and automatically build (and ideally test) the
latest version of the application from each source control change, such as Jenkins (jenkins.io).

The key handoff for CD is that the build system can invoke Ansible upon a successful build.
Users who also run unit or integration tests on code as a result of the build will also be one step ahead of the game.

Jenkins can utilize Tower to deploy the built artifact into multiple environments,

but a QA/stage environment modeled after production ups the ante and substantially improved predictability along the lifecycle. The data provided back by Ansible can then be referenced, and directly correlated to a Tower job in the Build Systems job.

Ansible’s unique multi-tier, multi-step orchestration capabilities, combined with its push-based architecture, allow for extremely rapid execution of these types of complex workflows

  • Ansible feature (note3)

MaxthonSnap20170324083511

  • What is Ansible?

 

  • Ansible 自動化組態技巧

 

 

(note1: www.ansible.com)

(note2: https://en.wikipedia.org/wiki/Ansible_(software))

(Why Ansible : https://www.ansible.com/it-automation)

(http://www.slideshare.net/joeywchou/ansible-73452143)

(note3: What is Ansible? // https://www.ansible.com/quick-start-video)

Google, AI, Machine Learning

Google cloud platform (note5)

Target : the enterprise cloud market

  • Position- To be a developer-friendly platform
  • Weakness

1) Not strong and no impression  on cloud service and the enterprise segment

2) No contribution to the open source community before.

(note1)

  • Stength : ASIC, GPU and TPU hardware in its cloud
  • Opportunity

1) begin to work with open source projects (note3)

  • Cloud Native Computing Foundation- the open-source container management tool

run by the Linux Foundation (note4)

Other partners in the new foundation include AT&T, Box, Cisco, Cloud Foundry Foundation, CoreOS, Cycle Computing, Docker, eBay, Goldman Sachs, Huawei, IBM, Intel, Joyent, Kismatic, Mesosphere, Red Hat, Switch SUPERNAP, Twitter, Univa, VMware and Weaveworks.

Absent : Microsoft, Amazon, Pivotal, and  Taiwanese tech companies)

the popular open source container orchestration system

  • TensorFlow for machine learning

-Spanner for launching massive distributed databases

-Draco for 3D graphics compression

2) To be a  developer-friendly platform

  • “OPENNESS" 

1)letting customers run whatever open source stack they choose on Google’s infrastructure,

2)releasing and supporting open source projects and making the ecosystem

3)the partners who build tools and technologies on top of GCP, a first class citizen on the platform.

4) treating them as part of the whole and the net is bringing the tech you want and using Google technology or using any of the [partner] services

The KSF :

1)Being open to win the mind shares of developers

2) much more supportive of the open source community makes people feel better about Google and makes developers feel better about working with their tools because they can avoid lock-in

  • Threat : peers: AWS (2006. 1st public cloud, market leader, 1st mover), Microsoft, IBM
  • Strategy

UsingKubernetes, the popular open source container orchestration system offer robust open source tools, something that surprised some people in this market.

  • 4 ways Google will enable enterprises to adopt machine learning and AI (note2)

1). Machine learning computing in Google Cloud

a deep learning algorithm can have tens of millions of parameters, training these machine learning models requires enormous computational resource

the Cloud Machine Learning Engine. This capability is designed for companies with data scientists and machine learning experts who are able to build their own unique machine learning models with libraries such as Tensorflow.

Google’s infrastructure as the solution to speed training times and improve the return on investment. Google has specialized ASIC, GPU and TPUhardware in its cloud to accelerate training and improve the ROI with on-demand cloud resource utilization. After the model is trained, it is deployed in range of platforms—from on-premise to mobile devices.

2. Algorithms and pretrained machine learning models

建ML model 需用 the machine learning engine, 用 Google’s pre-trained models (full list) using APIs to add machine learning capability to their applications, such as understanding natural language, images and natural language.

An API beta for understanding videos

demo: This 3-minute video of the demonstration of the Cloud Video Intelligence beta

3. Google acquires Kaggle for data

Google acquired Kaggle for data sets and talent. Kaggle, founded in 2010, is a community of 850,000 data scientists from around the world that hosts competitions to create the most accurate predictive models and market models, as well as to acquire new public data sets in a variety of fields.

4. Expertise

the Advanced Solutions Lab for customers with ambitious goals to develop machine learning to solve complex problems.

(note1: https://techcrunch.com/2017/03/09/google-in-the-cloud/)

(note2: http://www.networkworld.com/article/3179127/cloud-computing/4-ways-google-cloud-will-bring-ai-machine-learning-to-the-enterprise.html)

(note3:https://techcrunch.com/2015/07/21/as-kubernetes-hits-1-0-google-donates-technology-to-newly-formed-cloud-native-computing-foundation-with-ibm-intel-twitter-and-others/)

(note4: The mission of Linux foundation :The mission of this new foundation is to “help facilitate collaboration among developers and operators on common technologies for deploying cloud native applications and services,” )

(note5: https://cloud.google.com/)

(Reference : Google Cloud Platform 入門)

(Reference: https://technews.tw/tag/google-cloud-platform/)

 

Big data-related open source application

我有興趣的是1.4.5.6.  6 之前寫過. 遊戲業很早已有用Hadoop, 關鍵還是在應用的規模, 技術包山包海,我也只能挑重點看, 且戰且走,  Java Scirpt npm 裡有20幾萬個modules,  根本不可能線性學習, 必須博觀而約取.

1. Hadoop
OS:Windows、Linux 和 OS X
Reference : http://hadoop.apache.org

2.Hypertable

Hypertable 在互联网公司当中非常流行,它由谷歌开发,用来提高数据库的可扩展性。
与 Hadoop 兼容,提供商业支持和培训
OS:Linux 和 OS X
Reference:http://www.hypertable.com

3.Mesos

Apache Mesos 是一种资源抽象工具,有了它,企业就可以鼗整个数据中心当成一个资源池,它在又在运行 Hadoop、Spark 及类似应用程序的公司当中很流行
OS:Linux 和 OS X
Reference:http://mesos.apache.org

4.Presto

Presto 由 Facebook 开发,自称是“一款开源分布式 SQL 查询引擎,用于对大大小小(从 GB 级到 PB 级)的数据源运行交互式分析查询。”Facebook 表示,它将 Presto 用于对 300PB 大小的数据仓库执行查询

OS:Linux
Reference:https://prestodb.io

5. Solr

这种“快若闪电”的企业搜索平台声称高度可靠、扩展和容错

OS:与操作系统无关
Reference:http://Lucene.apache.org/solr/

6.Spark

Apache Spark 声称,“它在内存中运行程序的速度比 Hadoop MapReduce 最多快 100 倍,在磁盘上快 10 倍。

OS:Windows、Linux 和 OS X
Reference:http://spark.apache.org

7.Storm

Apache Storm 用来处理实时数据
OS:Linux
Reference:https://storm.apache.org

Apache Flink

這是個計算引擎, 號稱" 4G of Big Data" (note1), 快, 易用,開源, 效能佳, 但沒有儲存系統

  • Batch Processing
  • Interactive processing
  • Real-time stream processing
  • Graph Processing
  • Iterative Processing
  • In-memory processing

Flink is an alternative of Mapreduce, it processes data more than 100 times faster than MapReduce.

Flink is independant from hadoop but it can use hdfs to read, write, store, process the data. Flink does not provide its own data storage system.it takes data from distributed storage.

Flink  ecosystem:   (note2)

apache-flink-ecosystem-components

 Storage: 讀寫別家的資料庫大概都沒什麼問題

  • HDFS – Hadoop Distributed File System
  • Local-FS – Local File System
  • S3 – Simple Storage Service from Amazon
  • HBase – NoSQL Database in Hadoop ecosystem
  • MongoDB – NoSQL Database
  • RBDBMs – Any relational database
  • Kafka – Distributed messaging Queue
  • RabbitMQ – Messaging Queue
  • Flume – Data Collection and Aggregation Tool

以上都可

Deploy: 能分配部署資源 :

  • Local mode – On single node, in single JVM
  • Cluster – On multi-node cluster, with following resource manager
    • Standalone – This is the default resource manager which is shipped with Flink
    • YARN – This is very popular resource manager, it is part of Hadoop, introduced in Hadoop 2.x
    • Mesos – This is a generalized resource manager.
  • Cloud – on Amazon or Google cloud

Runtime :

the Distributed Streaming Dataflow, which is also called as kernel of Apache Flink. This is the core layer of flink which provides distributed processing, fault tolerance, reliability, native iterative processing capability, etc.

主從架構:

maxthonsnap20170216092524

 

特色:

  • Streaming – Flink is a true stream processing engine.
  • High performance – Flink’s data streaming Runtime provides very high throughput
  • Low latency – Flink can process the data in sub-second range without any delay
  • Event Time and Out-of-Order Events – Flink supports stream processing and windowing where events arrive delayed or out of order
  • Lightning fast speed – Flink processes data at lightning fast speed (hence also called as 4G of Big Data)
  • Fault Tolerance – Failure of hardware, node, software or a process doesn’t affect the cluster
  • Memory management – Flink works in managed memory and never get out of memory exception
  • Broad integration – Flink can be integrated with various storage system to process their data, it can be deployed with various resource management tools. It can also be integrated with several BI tools for reporting
  • Stream processing – Flink is a true streaming engine, can process live streams in sub-second interval
  • Program optimizer – Flink is shipped with an optimizer, before execution of a program it is optimized
  • Scalable – Flink is highly scalable. With increasing requirements we can scale flink cluster
  • Rich set of operators – Flink has lots of pre-defined operators to process the data. All the common operations can be done using these operators
  • Exactly-once Semantics – It can maintain custom state during computation
  • Highly flexible Streaming Windows – In flink we can customize windows by triggering conditions flexibly, to get required streaming patterns. We can create window according to time t1 to t5 and data driven windows.
  • Continuous streaming model with backpressure – Data streaming applications are executed with continuous (long lived) operators. Flink’s streaming engine naturally handles backpressure.
  • One Runtime for Streaming and Batch Processing – Batch processing and data streaming both have common runtime in flink
  • Easy and understandable Programmable APIs – Flink’s APIs are developed in a way to cover all the common operations, so programmers can use it efficiently.
  • Little tuning required – Requires no memory, network, serializer to configure

初看這Apache Flink, 電視台轉型需用到,以往直播用SNG車, 上衛星, 現在改串流技術,  光這樣成本就不知省多少,用途滿廣, 也可處理髒資料,推薦產品用, 作預測.

 

(note1: http://data-flair.training/blogs/apache-flink-production-fortune-500-companies-top-real-world-use-cases/)

(note2: data-flair.training/blogs/apache-flink-comprehensive-guide-tutorial-for-beginners/)

(Installation:

)

 

 

 

%d 位部落客按了讚: