18 Jul
2019
18 Jul
'19
11:35 p.m.
Resent the email at the mail thread.
On Fri, Jul 19, 2019 at 2:01 AM Ashish Sharma <sharma.ashish960(a)gmail.com>
wrote:
Hello everyone,
I am trying to implement xwiki helm chart in High availability but facing
some problems.
How we can implement High Availability(HA) in Kubernetes:-
1. Through Deployment:- In this approach we would use a shared file
storage, that would get attached to multiple pods(or containers)
2. Through StatefulSet:- In this approach every pod(or container) would
have it’s own different volume and state, which should be synchronised by
an consensus algorithm
Problem faced in xwiki case:-
1. In deployment case we need a locking mechanism for our filesystem,
moreover caching and states should also be maintained outside the container.
2. In StatefulSet the consensus algorithm is not present so it is
currently not possible.
Could anyone help me through it.
Thanks,
Ashish
On Wed, 10 Jul 2019 at 10:32 AM, Shubham Jain <jainshubham78(a)gmail.com>
wrote:
> Hey Ashish,
>
> I think Vincent/Thomas can help with the PR on the xwiki-docker project.
>
> Regarding adding docs for High-Availability setup using shared data
> volumes, I think you can use could create a docs folder in the GitHub repo,
> and add the docs in Markdown format. I think you can also document it in
> XWiki blog <https://www.xwiki.org/xwiki/bin/view/Blog/>. The chart
> should support enabling HA, and then passing a storage class which supports
> shareable volumes to Pods, and then you can document examples for using
> this with something like Rook+GKE setup.
>
> Best,
> Shubham.
>
>
> > On 08-Jul-2019, at 1:57 PM, Ashish Sharma <sharma.ashish960(a)gmail.com>
> wrote:
> >
> > Hello everyone,
> >
> > I require the xwiki solr docker container to do progress with my work. I
> > have made a pull request
> > <https://github.com/xwiki-contrib/docker-xwiki/pull/22>, for the same.
> > Please look into it.
> >
> > Moreover, I was working with Rook(Ceph Operator) for a shared file
> system
> > which would be a better approach to deploy our app on production. So I
> > think we should document it somewhere. Can anyone help me where should
> I do
> > that. It may contain the details to configure rook-ceph, and then
> > connecting it.
> >
> > P.S. I am writing a blog at Medium on how to configure rook on Google
> > Kubernetes Engine. We could also use that?
> >
> > Thanks,
> > Ashish
> >
> > On Mon, Jun 24, 2019 at 8:31 PM Ashish Sharma <
> sharma.ashish960(a)gmail.com>
> > wrote:
> >
> >> Hi Shubham,
> >>
> >> Thank you Shubham, I will start working on adding test cases.
> >>
> >> Thanks,
> >> Ashish
> >>
> >> On Mon, 24 Jun 2019 at 3:32 PM, Shubham Jain
<jainshubham78(a)gmail.com>
> >> wrote:
> >>
> >>> Hey Ashish,
> >>>
> >>> Thanks for the update.
> >>>
> >>> Adding our chart to incubator section of helm community charts
> repository
> >>> sounds great. Although (unlike Dockerhub) this repo is for community
> charts
> >>> and not really official charts of the respective installations, but
> >>> supporting this chart in the community chart repo should help with the
> >>> visibility.
> >>>
> >>> Lets also work on adding tests to the helm chart to ensure things are
> >>> going fine.
> >>>
> >>> Best,
> >>> Shubham.
> >>>
> >>>> On 21-Jun-2019, at 4:37 AM, Ashish Sharma <
> sharma.ashish960(a)gmail.com>
> >>> wrote:
> >>>>
> >>>> Hi everyone,
> >>>>
> >>>> Update on HELM project:-
> >>>>
> >>>> We have added some features to the chart that includes:-
> >>>> * Making chart configurable with both mysql and postgres database
> >>>> * We are now using configmaps and secrets for storing database
> >>> credentials
> >>>> * We use helm dependency to manage the dependencies of the chart
> >>>>
> >>>> I think our chart is ready to be deployed in the incubator section
of
> >>> helm
> >>>> community repo. <https://github.com/helm/charts> If you guys
agree I
> >>> would
> >>>> make the necessary changes required to deploy it there.
> >>>>
> >>>> Thanks
> >>>> - Ashish
> >>>>
> >>>> On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol
<vincent(a)massol.net>
> >>> wrote:
> >>>>
> >>>>> Hi Ashish,
> >>>>>
> >>>>>> On 5 Jun 2019, at 15:07, Ashish Sharma
<sharma.ashish960(a)gmail.com
> >
> >>>>> wrote:
> >>>>>>
> >>>>>> Hi everyone,
> >>>>>>
> >>>>>> The first week update of helm chart xwiki project. We
currently
> have a
> >>>>>> chart that would deploy xwiki mysql stateful service with a
> persistent
> >>>>>> volume. The link to the repo:-
> >>>>>>
> >>>>>> https://github.com/xwiki-contrib/xwiki-helm
> >>>>>
> >>>>> Thanks for the update.
> >>>>>
> >>>>> Would be great if you could update the README file to explain
who to
> >>> use
> >>>>> it.
> >>>>>
> >>>>> Thanks
> >>>>> -Vincent
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> Thank You
> >>>>>> Ashish
> >>>>>>
> >>>>>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <
> >>>>> sharma.ashish960(a)gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <
> vincent(a)massol.net>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Ashish,
> >>>>>>>>
> >>>>>>>>> On 21 May 2019, at 19:37, Ashish Sharma <
> >>> sharma.ashish960(a)gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi everyone,
> >>>>>>>>> Thanks for having me here
> >>>>>>>>>
> >>>>>>>>> About Me
> >>>>>>>>>
> >>>>>>>>> I am Ashish Sharma, selected as a student for
Google Summer of
> >>> Code. I
> >>>>>>>> am
> >>>>>>>>> final year student enrolled in Guru Gobind Singh
Indraprastha
> >>>>>>>> University,
> >>>>>>>>> Delhi. I am a resident of India.
> >>>>>>>>>
> >>>>>>>>> Profiles
> >>>>>>>>> GitHub -
https://github.com/ashish932/xwiki-helm-chart/
> >>>>>>>>> LinkedIn -
https://www.linkedin.com/in/ashish932/
> >>>>>>>>> Riot - @ashish932:matrix.org
> >>>>>>>>>
> >>>>>>>>> I will be presenting my project "Helm Chart
for XWiki" to all of
> >>> you.
> >>>>>>>>> Following
> >>>>>>>>> are the relevant details.
> >>>>>>>>>
> >>>>>>>>> Helm Chart for XWiki
> >>>>>>>>>
> >>>>>>>>> Mentors: Shubham Jain, Neha Gupta
> >>>>>>>>>
> >>>>>>>>> Technologies: Kubernetes, Docker, other if
required
> >>>>>>>>>
> >>>>>>>>> Overview
> >>>>>>>>> The proposed project is a helm chart that would
deploy xwiki as
> >>> highly
> >>>>>>>>> available and reliable. It should be
configurable with different
> >>>>>>>>> databases(either a standalone database or a
clustered one) that
> are
> >>>>>>>>> configurable with xwiki. It would give the
option to either
> >>> configure
> >>>>>>>> solr
> >>>>>>>>> externally (standalone or clustered) or managed
within the
> >>> container.
> >>>>>>>>
> >>>>>>>> It also needs to be deployed on various Servlet
Containers and
> with
> >>> a
> >>>>>>>> LibreOffice server configured.
> >>>>>>>>
> >>>>>>>> In short, it should support all the configurations
supported by
> the
> >>>>>>>> official XWiki Docker images :)
> >>>>>>>>
> >>>>>>>>> It
> >>>>>>>>> should deploy the app on a shared file system
like a rook. It
> >>> should
> >>>>>>>>> support Istio virtual services, istio matrix,
and istio
> distributed
> >>>>>>>> tracing
> >>>>>>>>> and should be a secured system with RBAC and
security credential
> >>>>>>>> rotation.
> >>>>>>>>
> >>>>>>>> Ok I have no knowledge about this… The shared file
system sounds
> >>> good
> >>>>> to
> >>>>>>>> support clustering XWiki for the permanent directory
sharing.
> >>>>>>>>
> >>>>>>>>> The chart should be easily deployed on GKE and
amazon EKS.
> >>>>>>>>
> >>>>>>>> On locally on minikube for testing I guess?
> >>>>>>>>
> >>>>>>>
> >>>>>>> Yes we can test using minikube one
> >>>>>>>
> >>>>>>>>
> >>>>>>>>> Features
> >>>>>>>>>
> >>>>>>>>> -> Support for different Databases
> >>>>>>>>> -> Choice between using an external
database, a single node DB
> or
> >>> a
> >>>>>>>>> multi-cluster DB setup
> >>>>>>>>> -> Support for shared file system
> >>>>>>>>> -> Support for istio and it's services
> >>>>>>>>> -> RBAC, SSL and other security methods
> >>>>>>>>>
> >>>>>>>>> If you have any features in mind that should be
added please
> feel
> >>> free
> >>>>>>>> to
> >>>>>>>>> reply to this mail.
> >>>>>>>>>
> >>>>>>>>> Some Design Questions?
> >>>>>>>>> -> Which Databases should be supported?
> >>>>>>>>
> >>>>>>>> See
> >>>>>>>>
> >>>>>
> >>>
>
https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupp…
> >>>>>>>>
> >>>>>>>> More generally see the XWiki Docker image
documentation and
> >>>>>>>>
https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
> >>>>>>>>
> >>>>>>>>> -> As we have to detach solr out of the
docker container(run it
> in
> >>> an
> >>>>>>>>> independent container) would be there a
requirement for a code
> >>> change,
> >>>>>>>> and
> >>>>>>>>> we should approach it?
> >>>>>>>>
> >>>>>>>> See the doc for Solr which explains how to configure
an external
> >>> SOLR.
> >>>>>>>> There are some XWiki code/config to deploy in the
external SOLR
> >>> AFAIR.
> >>>>>>>> comunity controls chart
> >>>>>>>>> -> Apart from solr is there any other
stateful service that
> could
> >>> or
> >>>>>>>> should
> >>>>>>>>> be detached from the docker container?
> >>>>>>>>
> >>>>>>>> Which docker container are you referring to? The
Servlet
> Container
> >>> one?
> >>>>>>>>
> >>>>>>>> xwiki:lts-mysql-tomcat
> >>>>>>>
> >>>>>>>
> >>>>>>>>> Here is my current repository which deploys
XWiki for MySQL
> >>> database
> >>>>>>>> using
> >>>>>>>>> official XWiki docker container:-
> >>>>>>>>> https://github.com/ashish932/xwiki-helm-chart/
> >>>>>>>>
> >>>>>>>> Will need to have a look. Some additional
questions:
> >>>>>>>>
> >>>>>>>> 1) Is there an official place for publishing helm
charts,
> similar to
> >>>>>>>> dockerhub for docker images? How do you provide the
helm charts
> to
> >>>>> XWiki
> >>>>>>>> users in term of packaging? In other words, what
will we/you
> >>> document
> >>>>> on
> >>>>>>>>
> >>>>>
> >>>
> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
> >>>>>>>> ?
> >>>>>>>>
> >>>>>>>> Yes, There is a centralized community control chart
management
> >>>>>>> repository. Helm charts are hosted there. Organizations
like
> mysql,
> >>>>>>> wordpress uses it.
> >>>>>>> https://github.com/helm/charts
> >>>>>>> But in addition to that, we can build our own charts
repository
> >>> locally,
> >>>>>>> and host it. Similar to docker run command, we have helm
update
> >>> command
> >>>>>>>
> >>>>>>>
> >>>>>>>> 2) How do you plan to have smoke tests for the XWiki
Helm
> charts, as
> >>>>> part
> >>>>>>>> of the build, to verify that they work fine? How can
we execute
> >>> XWik’s
> >>>>>>>> functional tests based on TestContainers using Helm
Charts?
> >>>>>>>>
> >>>>>>>
> >>>>>>> There are some tools available
> >>>>>>> chart-testing(started by helm) -98 commits
> >>>>>>> terratest - 1039 commits (It looks a good tool to use)
> >>>>>>>
> >>>>>>>
> >>>>>>>> 3) Do the helm charts for XWiki need to be built? If
so, can it
> be
> >>> done
> >>>>>>>> with Maven? Since we’ll be offering several I guess,
how do you
> >>> avoid
> >>>>> code
> >>>>>>>> duplication and make maintenance of them as simple
as possible?
> >>>>>>>>
> >>>>>>>
> >>>>>>> No, the charts don’t need to be built externally, though
it gets
> >>>>>>> compressed when pushed f we build to chart
repository(this is an
> >>>>> automatic
> >>>>>>> step in case of official chart repo and we can automate
using ci
> our
> >>> own
> >>>>>>> chart repo)
> >>>>>>>
> >>>>>>> 4) Could you provide a roadmap with various milestones
and dates?
> See
> >>>>> the
> >>>>>>>> last roadmap email I sent on the devs list and
please reply to
> it.
> >>>>>>>>
> >>>>>>>
> >>>>>>> Okay I would work on that and update the progress on the
mail
> thread
> >>>>>>>
> >>>>>>>
> >>>>>>>> 5) It seems you missed the bonding period so you’ll
need to catch
> >>> up on
> >>>>>>>> that. Could you pick some jira issue and work on
them quickly?
> This
> >>> is
> >>>>> to
> >>>>>>>> make sure you use the proper xwiki communication
channels, learn
> to
> >>>>> work
> >>>>>>>> with the xwiki team and start understanding the
xwiki
> architecture,
> >>>>> and how
> >>>>>>>> extensions are developed for XWiki. Since you’re a
bit late,
> you’ll
> >>>>> need to
> >>>>>>>> catch up and achieve this very quickly now and be
present full
> time
> >>> on
> >>>>> the
> >>>>>>>> xwiki chat :)
> >>>>>>>>
> >>>>>>> I would start doing that
> >>>>>>>
> >>>>>>>>
> >>>>>>>> 6) Idea: please review the issues on the XWiki
Docker project’s
> >>> jira.
> >>>>>>>> Maybe you can help with those too as part of
onboarding/bonding.
> It
> >>>>> should
> >>>>>>>> also give you additional use cases for the XWiki
Helm Charts IMO.
> >>>>>>>>
> >>>>>>>> Thanks! I’m eager to follow your work and see xwiki
helm charts
> in
> >>>>> action.
> >>>>>>>>
> >>>>>>>> -Vincent
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Thank You
> >>>>>>>>> Ashish Sharma
> >>>>>>>>
> >>>>>>>> Thank You
> >>>>>>> Ashish
> >>>>>>>
> >>>>>
> >>>>>
> >>>
> >>>
>
>