Re: [xwiki-devs] Xwiki Helm HA

On 20-Jul-2019, at 6:36 AM, Ashish Sharma &lt;sharma.ashish960(a)gmail.com&gt; wrote: Hey Shubham, I have tried initialising with 1 pod and scaling it up to multiple pods, also tried session affinity, and some other cases but none of them worked. Thanks, Ashish On Fri, 19 Jul 2019 at 4:51 PM, Shubham Jain &lt;jainshubham78(a)gmail.com&gt; wrote: > Hey Ashish, > > I don’t understand XWiki well enough to fully understand the implications > of running a clustered setup by sharing file system. > > Can you try keeping the deployment size initially 1, and then later try to > increase the size? Does that also cause problems? > > Best, > Shubham. > >> On 19-Jul-2019, at 4:03 PM, Ashish Sharma &lt;sharma.ashish960(a)gmail.com&gt; > wrote: >> >> Hey Shubham, >> >> I have moved the solr folder as exposed by the docker container to the > file >> system. I think in-memory cache is creating problem. >> >> I started 3 container of xwiki with shared file storage, but all three > were >> trying to initiate the postgress database, that is causing problem. >> >> I want to know few things about xwiki:- >> >> 1. Do we use java threads ? -> if we do then filesystem locking should > have >> been implemented for that? >> >> 2. Other stateful things in our application like caching or other >> filestorage that should be externalise? >> >> >> Thanks, >> Ashish >> >> On Fri, 19 Jul 2019 at 3:59 AM, Shubham Jain &lt;jainshubham78(a)gmail.com&gt; >> wrote: >> >>> Hey Ashish, >>> >>> I think for now you could stick to deployment. Have you been ever to >>> discover the data you need to move to the shared file system? Is the >>> in-memory caching mechanism causing problems? >>> >>> @Thomas/Vincent - Can you please help us understand the implications of >>> running XWiki in a clustered mode by sharing data volumes? >>> >>> Best, >>> Shubham. >>> >>> >>> Best, >>> Shubham? >>> >>>> On 19-Jul-2019, at 5:35 AM, Ashish Sharma &lt;sharma.ashish960(a)gmail.com&gt; >>> wrote: >>>> >>>> Resent the email at the mail thread. >>>> >>>> On Fri, Jul 19, 2019 at 2:01 AM Ashish Sharma < >>> sharma.ashish960(a)gmail.com&gt; >>>> wrote: >>>> >>>>> Hello everyone, >>>>> >>>>> I am trying to implement xwiki helm chart in High availability but >>> facing >>>>> some problems. >>>>> >>>>> How we can implement High Availability(HA) in Kubernetes:- >>>>> >>>>> 1. Through Deployment:- In this approach we would use a shared file >>>>> storage, that would get attached to multiple pods(or containers) >>>>> 2. Through StatefulSet:- In this approach every pod(or container) > would >>>>> have it’s own different volume and state, which should be synchronised >>> by >>>>> an consensus algorithm >>>>> >>>>> Problem faced in xwiki case:- >>>>> >>>>> 1. In deployment case we need a locking mechanism for our filesystem, >>>>> moreover caching and states should also be maintained outside the >>> container. >>>>> 2. In StatefulSet the consensus algorithm is not present so it is >>>>> currently not possible. >>>>> >>>>> Could anyone help me through it. >>>>> >>>>> Thanks, >>>>> Ashish >>>>> >>>>> On Wed, 10 Jul 2019 at 10:32 AM, Shubham Jain < > jainshubham78(a)gmail.com&gt; >>>>> wrote: >>>>> >>>>>> Hey Ashish, >>>>>> >>>>>> I think Vincent/Thomas can help with the PR on the xwiki-docker >>> project. >>>>>> >>>>>> Regarding adding docs for High-Availability setup using shared data >>>>>> volumes, I think you can use could create a docs folder in the GitHub >>> repo, >>>>>> and add the docs in Markdown format. I think you can also document it >>> in >>>>>> XWiki blog <https://www.xwiki.org/xwiki/bin/view/Blog/>. The chart >>>>>> should support enabling HA, and then passing a storage class which >>> supports >>>>>> shareable volumes to Pods, and then you can document examples for > using >>>>>> this with something like Rook+GKE setup. >>>>>> >>>>>> Best, >>>>>> Shubham. >>>>>> >>>>>> >>>>>>> On 08-Jul-2019, at 1:57 PM, Ashish Sharma < > sharma.ashish960(a)gmail.com >>>> >>>>>> wrote: >>>>>>> >>>>>>> Hello everyone, >>>>>>> >>>>>>> I require the xwiki solr docker container to do progress with my >>> work. I >>>>>>> have made a pull request >>>>>>> <https://github.com/xwiki-contrib/docker-xwiki/pull/22>, for the >>> same. >>>>>>> Please look into it. >>>>>>> >>>>>>> Moreover, I was working with Rook(Ceph Operator) for a shared file >>>>>> system >>>>>>> which would be a better approach to deploy our app on production. > So I >>>>>>> think we should document it somewhere. Can anyone help me where > should >>>>>> I do >>>>>>> that. It may contain the details to configure rook-ceph, and then >>>>>>> connecting it. >>>>>>> >>>>>>> P.S. I am writing a blog at Medium on how to configure rook on > Google >>>>>>> Kubernetes Engine. We could also use that? >>>>>>> >>>>>>> Thanks, >>>>>>> Ashish >>>>>>> >>>>>>> On Mon, Jun 24, 2019 at 8:31 PM Ashish Sharma < >>>>>> sharma.ashish960(a)gmail.com&gt; >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Shubham, >>>>>>>> >>>>>>>> Thank you Shubham, I will start working on adding test cases. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Ashish >>>>>>>> >>>>>>>> On Mon, 24 Jun 2019 at 3:32 PM, Shubham Jain < >>> jainshubham78(a)gmail.com&gt; >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hey Ashish, >>>>>>>>> >>>>>>>>> Thanks for the update. >>>>>>>>> >>>>>>>>> Adding our chart to incubator section of helm community charts >>>>>> repository >>>>>>>>> sounds great. Although (unlike Dockerhub) this repo is for > community >>>>>> charts >>>>>>>>> and not really official charts of the respective installations, > but >>>>>>>>> supporting this chart in the community chart repo should help with >>> the >>>>>>>>> visibility. >>>>>>>>> >>>>>>>>> Lets also work on adding tests to the helm chart to ensure things >>> are >>>>>>>>> going fine. >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Shubham. >>>>>>>>> >>>>>>>>>> On 21-Jun-2019, at 4:37 AM, Ashish Sharma < >>>>>> sharma.ashish960(a)gmail.com&gt; >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi everyone, >>>>>>>>>> >>>>>>>>>> Update on HELM project:- >>>>>>>>>> >>>>>>>>>> We have added some features to the chart that includes:- >>>>>>>>>> * Making chart configurable with both mysql and postgres database >>>>>>>>>> * We are now using configmaps and secrets for storing database >>>>>>>>> credentials >>>>>>>>>> * We use helm dependency to manage the dependencies of the chart >>>>>>>>>> >>>>>>>>>> I think our chart is ready to be deployed in the incubator > section >>> of >>>>>>>>> helm >>>>>>>>>> community repo. <https://github.com/helm/charts> If you guys >>> agree I >>>>>>>>> would >>>>>>>>>> make the necessary changes required to deploy it there. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> - Ashish >>>>>>>>>> >>>>>>>>>> On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol < > vincent(a)massol.net&gt; >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Ashish, >>>>>>>>>>> >>>>>>>>>>>> On 5 Jun 2019, at 15:07, Ashish Sharma < >>> sharma.ashish960(a)gmail.com >>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi everyone, >>>>>>>>>>>> >>>>>>>>>>>> The first week update of helm chart xwiki project. We currently >>>>>> have a >>>>>>>>>>>> chart that would deploy xwiki mysql stateful service with a >>>>>> persistent >>>>>>>>>>>> volume. The link to the repo:- >>>>>>>>>>>> >>>>>>>>>>>> https://github.com/xwiki-contrib/xwiki-helm >>>>>>>>>>> >>>>>>>>>>> Thanks for the update. >>>>>>>>>>> >>>>>>>>>>> Would be great if you could update the README file to explain > who >>> to >>>>>>>>> use >>>>>>>>>>> it. >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> -Vincent >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thank You >>>>>>>>>>>> Ashish >>>>>>>>>>>> >>>>>>>>>>>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma < >>>>>>>>>>> sharma.ashish960(a)gmail.com&gt; >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol < >>>>>> vincent(a)massol.net&gt; >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Ashish, >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 21 May 2019, at 19:37, Ashish Sharma < >>>>>>>>> sharma.ashish960(a)gmail.com&gt; >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>>>> Thanks for having me here >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> About Me >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I am Ashish Sharma, selected as a student for Google Summer > of >>>>>>>>> Code. I >>>>>>>>>>>>>> am >>>>>>>>>>>>>>> final year student enrolled in Guru Gobind Singh > Indraprastha >>>>>>>>>>>>>> University, >>>>>>>>>>>>>>> Delhi. I am a resident of India. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Profiles >>>>>>>>>>>>>>> GitHub - https://github.com/ashish932/xwiki-helm-chart/ >>>>>>>>>>>>>>> LinkedIn - https://www.linkedin.com/in/ashish932/ >>>>>>>>>>>>>>> Riot - @ashish932:matrix.org >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I will be presenting my project "Helm Chart for XWiki" to > all >>> of >>>>>>>>> you. >>>>>>>>>>>>>>> Following >>>>>>>>>>>>>>> are the relevant details. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Helm Chart for XWiki >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Mentors: Shubham Jain, Neha Gupta >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Technologies: Kubernetes, Docker, other if required >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Overview >>>>>>>>>>>>>>> The proposed project is a helm chart that would deploy xwiki >>> as >>>>>>>>> highly >>>>>>>>>>>>>>> available and reliable. It should be configurable with >>> different >>>>>>>>>>>>>>> databases(either a standalone database or a clustered one) >>> that >>>>>> are >>>>>>>>>>>>>>> configurable with xwiki. It would give the option to either >>>>>>>>> configure >>>>>>>>>>>>>> solr >>>>>>>>>>>>>>> externally (standalone or clustered) or managed within the >>>>>>>>> container. >>>>>>>>>>>>>> >>>>>>>>>>>>>> It also needs to be deployed on various Servlet Containers > and >>>>>> with >>>>>>>>> a >>>>>>>>>>>>>> LibreOffice server configured. >>>>>>>>>>>>>> >>>>>>>>>>>>>> In short, it should support all the configurations supported > by >>>>>> the >>>>>>>>>>>>>> official XWiki Docker images :) >>>>>>>>>>>>>> >>>>>>>>>>>>>>> It >>>>>>>>>>>>>>> should deploy the app on a shared file system like a rook. > It >>>>>>>>> should >>>>>>>>>>>>>>> support Istio virtual services, istio matrix, and istio >>>>>> distributed >>>>>>>>>>>>>> tracing >>>>>>>>>>>>>>> and should be a secured system with RBAC and security >>> credential >>>>>>>>>>>>>> rotation. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Ok I have no knowledge about this… The shared file system >>> sounds >>>>>>>>> good >>>>>>>>>>> to >>>>>>>>>>>>>> support clustering XWiki for the permanent directory sharing. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> The chart should be easily deployed on GKE and amazon EKS. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On locally on minikube for testing I guess? >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Yes we can test using minikube one >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Features >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -> Support for different Databases >>>>>>>>>>>>>>> -> Choice between using an external database, a single node >>> DB >>>>>> or >>>>>>>>> a >>>>>>>>>>>>>>> multi-cluster DB setup >>>>>>>>>>>>>>> -> Support for shared file system >>>>>>>>>>>>>>> -> Support for istio and it's services >>>>>>>>>>>>>>> -> RBAC, SSL and other security methods >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If you have any features in mind that should be added please >>>>>> feel >>>>>>>>> free >>>>>>>>>>>>>> to >>>>>>>>>>>>>>> reply to this mail. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Some Design Questions? >>>>>>>>>>>>>>> -> Which Databases should be supported? >>>>>>>>>>>>>> >>>>>>>>>>>>>> See >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>> >>> > https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupp… >>>>>>>>>>>>>> >>>>>>>>>>>>>> More generally see the XWiki Docker image documentation and >>>>>>>>>>>>>> > https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy >>>>>>>>>>>>>> >>>>>>>>>>>>>>> -> As we have to detach solr out of the docker container(run >>> it >>>>>> in >>>>>>>>> an >>>>>>>>>>>>>>> independent container) would be there a requirement for a > code >>>>>>>>> change, >>>>>>>>>>>>>> and >>>>>>>>>>>>>>> we should approach it? >>>>>>>>>>>>>> >>>>>>>>>>>>>> See the doc for Solr which explains how to configure an >>> external >>>>>>>>> SOLR. >>>>>>>>>>>>>> There are some XWiki code/config to deploy in the external > SOLR >>>>>>>>> AFAIR. >>>>>>>>>>>>>> comunity controls chart >>>>>>>>>>>>>>> -> Apart from solr is there any other stateful service that >>>>>> could >>>>>>>>> or >>>>>>>>>>>>>> should >>>>>>>>>>>>>>> be detached from the docker container? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Which docker container are you referring to? The Servlet >>>>>> Container >>>>>>>>> one? >>>>>>>>>>>>>> >>>>>>>>>>>>>> xwiki:lts-mysql-tomcat >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>>> Here is my current repository which deploys XWiki for MySQL >>>>>>>>> database >>>>>>>>>>>>>> using >>>>>>>>>>>>>>> official XWiki docker container:- >>>>>>>>>>>>>>> https://github.com/ashish932/xwiki-helm-chart/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Will need to have a look. Some additional questions: >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1) Is there an official place for publishing helm charts, >>>>>> similar to >>>>>>>>>>>>>> dockerhub for docker images? How do you provide the helm > charts >>>>>> to >>>>>>>>>>> XWiki >>>>>>>>>>>>>> users in term of packaging? In other words, what will we/you >>>>>>>>> document >>>>>>>>>>> on >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>> >>> > https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/ >>>>>>>>>>>>>> ? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yes, There is a centralized community control chart > management >>>>>>>>>>>>> repository. Helm charts are hosted there. Organizations like >>>>>> mysql, >>>>>>>>>>>>> wordpress uses it. >>>>>>>>>>>>> https://github.com/helm/charts >>>>>>>>>>>>> But in addition to that, we can build our own charts > repository >>>>>>>>> locally, >>>>>>>>>>>>> and host it. Similar to docker run command, we have helm > update >>>>>>>>> command >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> 2) How do you plan to have smoke tests for the XWiki Helm >>>>>> charts, as >>>>>>>>>>> part >>>>>>>>>>>>>> of the build, to verify that they work fine? How can we > execute >>>>>>>>> XWik’s >>>>>>>>>>>>>> functional tests based on TestContainers using Helm Charts? >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> There are some tools available >>>>>>>>>>>>> chart-testing(started by helm) -98 commits >>>>>>>>>>>>> terratest - 1039 commits (It looks a good tool to use) >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> 3) Do the helm charts for XWiki need to be built? If so, can > it >>>>>> be >>>>>>>>> done >>>>>>>>>>>>>> with Maven? Since we’ll be offering several I guess, how do > you >>>>>>>>> avoid >>>>>>>>>>> code >>>>>>>>>>>>>> duplication and make maintenance of them as simple as > possible? >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> No, the charts don’t need to be built externally, though it > gets >>>>>>>>>>>>> compressed when pushed f we build to chart repository(this is > an >>>>>>>>>>> automatic >>>>>>>>>>>>> step in case of official chart repo and we can automate using > ci >>>>>> our >>>>>>>>> own >>>>>>>>>>>>> chart repo) >>>>>>>>>>>>> >>>>>>>>>>>>> 4) Could you provide a roadmap with various milestones and >>> dates? >>>>>> See >>>>>>>>>>> the >>>>>>>>>>>>>> last roadmap email I sent on the devs list and please reply > to >>>>>> it. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Okay I would work on that and update the progress on the mail >>>>>> thread >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> 5) It seems you missed the bonding period so you’ll need to >>> catch >>>>>>>>> up on >>>>>>>>>>>>>> that. Could you pick some jira issue and work on them > quickly? >>>>>> This >>>>>>>>> is >>>>>>>>>>> to >>>>>>>>>>>>>> make sure you use the proper xwiki communication channels, >>> learn >>>>>> to >>>>>>>>>>> work >>>>>>>>>>>>>> with the xwiki team and start understanding the xwiki >>>>>> architecture, >>>>>>>>>>> and how >>>>>>>>>>>>>> extensions are developed for XWiki. Since you’re a bit late, >>>>>> you’ll >>>>>>>>>>> need to >>>>>>>>>>>>>> catch up and achieve this very quickly now and be present > full >>>>>> time >>>>>>>>> on >>>>>>>>>>> the >>>>>>>>>>>>>> xwiki chat :) >>>>>>>>>>>>>> >>>>>>>>>>>>> I would start doing that >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> 6) Idea: please review the issues on the XWiki Docker > project’s >>>>>>>>> jira. >>>>>>>>>>>>>> Maybe you can help with those too as part of >>> onboarding/bonding. >>>>>> It >>>>>>>>>>> should >>>>>>>>>>>>>> also give you additional use cases for the XWiki Helm Charts >>> IMO. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks! I’m eager to follow your work and see xwiki helm > charts >>>>>> in >>>>>>>>>>> action. >>>>>>>>>>>>>> >>>>>>>>>>>>>> -Vincent >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thank You >>>>>>>>>>>>>>> Ashish Sharma >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thank You >>>>>>>>>>>>> Ashish >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>> >>>>>> >>> >