Similar project

[luxas]

If you guys haven't already noticed, I have a similar project.
It has been up and working on Github since 1st October.

https://github.com/luxas/kubernetes-on-arm

It supports addons, multiple boards and is running on Arch Linux ARM by default and soon HypriotOS too OOTB.
Do you have some comments on it?
You could also swap andrewpsuedonym/etcd:2.1.1 (637 MB) for the newer kubernetesonarm/etcd (which is v2.2.0 and 608 MB smaller)
The same for andrewpsuedonym/flannel and kubernetesonarm/flannel

I'm glad you are interested in k8s on ARM 😄

[brendandburns]

I think that we should probably just probably consolidate the two sites (and any others) move this to github.com/kubernetes/k8s-arm or somesuch.

@luxas @awassink what do you think? Would you both be ok being maintainers for that project in the Kubernetes org instead of your own personal repos?

I think it makes sense to collaborate in one single place that is easy to find...

I'm happy to create the repo and add you folks as maintainers if that's agreeable.

[awassink]

@luxas thanx for notifying! You have done a lot of work already! Since your comment I see in repository that you in the meantime already added hypriot support. Perfect.

My intention was to make a .deb package to improve the installation (and update) process. I see your working on that also.

What do you think of @brendandburns invite? Your project is a very good base for that.

[luxas]

Thanks for the invite @brendandburns!
Yes, it makes sense to have it in the Kubernetes organization. But I really think most of this work could be merged into mainline k8s. As you know I have a proposal open for that. Just now, I’m working on merging my work with Kubernetes. This is what I think should be done:

• Get docker-multinode working really well: Update hyperkube apiserver flag: --address kubernetes/kubernetes#19147 Update docker-multinode instructions and version kubernetes/kubernetes#18894 Enable serviceaccounts for the docker-multinode deployment kubernetes/kubernetes#17213 Run kubelet in dockerized clusters via nsenter. kubernetes/kubernetes#19069
• Fix and harmonize all docker image Makefiles to have the gcr.io/google_containers/binary-arch:version tag and with ARM base images. I will soon upload a PR for that.
• Include flannel in cluster/images/ as how etcd is implemented today
• Get kubernetes cluster deployment & management using docker containers kubernetes/kubernetes#13901 merged and customize it for ARM
• Include the ability to build hyperkube dynamically for ARMv6 on amd64 hosts. I’ve a script for that already: https://gist.github.com/luxas/e319b314f22b18243b1d and I’m working on merging that with kube-build
• Build etcd and flannel with kube-build and maybe store them in: https://storage.googleapis.com/kubernetes-release/release/${VERSION}/bin/linux/${ARCH}/(etcd|flannel) if everyone agrees or include etcd and flannel in hyperkube (even better): Provide an 'all-in-one' binary for server components (hyperkube) kubernetes/kubernetes#5755
• Include documentation for ARM in docs/getting-started-guides/ARM.md and docs/getting-started-guides/ARM/
• Push hyperkube as a part of the release process (related: Push server docker images during the release (attempt 2) kubernetes/kubernetes#19061)

The reason why I have my project is that it takes quite long time for code to get into Kubernetes mainline, so it’s much faster to have a personal project for hacking on this.

@brendandburns I would really appreciate it if you merged kubernetes/kubernetes#17213, I don’t understand why it has been pending for so long

@awassink
Yes, my project is the most developed for the time being. I’ve got a very nice response from other interested people, they are really excited about this. Since v0.6.2 of my project, I also have a .deb package which works for example on main Raspbian, so it’s very easy to get k8s for everyone.

It features:

• Only systemd as a dependency.
• kubernetesonarm/etcd, flannel, hyperkube, kube2sky, pause, registry, skydns images published on the Hub.
• Binaries uploaded to Github
• ServiceAccount support => dns addon
• registry addon
• A script: kube-config which starts/stops k8s master/worker, starts/stops addons, installs all required things on first use, information about k8s and the node and an ability to remove all Kubernetes data. This is the really important thing in my project
• SD Card creation for HypriotOS and Arch Linux ARM, which includes the project at the same time
• OOTB support for: Raspberry Pi 1 and 2, Cubietruck, Parallella, Banana Pro + all boards that may run the .deb file

I don’t know how much you have inspected my project, so wrote it here also.

So, I’m interested in creating a kubernetes/k8s-on-arm project, but there is a bunch of questions.
Do we have a better place than this to discuss? Maybe sig-arm or something?

Example questions: Where should the binaries be uploaded? Who should be the responsible maintainer? Who should build the binaries and when? How about versioning? Which features should it have? What should the directory structure look like? How should updates to main Kubernetes be reflected?

Oh, that text ended up quite long. :)
I hope you get me right. If we do this, I just want to make sure we do it right.
I’m a bit divided. I think this could be done in main Kubernetes (better long-term) and until that my project works well. On the other hand, it would be fun to make this official kubernetes/k8s-on-arm project too.

But none of us have 30 hours a day, and for me, this is just one of my hobbies.
So we should discuss these things first. Maybe Brendan could create sig-arm on Slack?
Another option (if you agree) is to create the project only with a link to luxas/kubernetes-on-arm and if you like, you could send PRs.
Have a good day!