Re: [xwiki-devs] Map Application - GSoC 19

Hi Fawad, Caty, all, Good to know where you are, Fawad. You're on the right track indeed imho, we're in tune. I'm sending you below some feedback (part of it is code review that could take place directly on GitHub, but I feel it easier to discuss about it all together on the list for now, let's see how it goes, I'm open to any suggestion). == MapClass and MapTemplate == - Query field: I'd suggest we use a large string field because the queries could become more complex in the future - I would consider putting the default values directly in the template, so that the user can know what they are when he creates a map, what do you think? - What about using an integer rather than a long for the zoom level? - You probably already have this in mind: this will be handy to create a MapTemplateProvider as well, for example as indicated on < https://extensions.xwiki.org/xwiki/bin/view/Extension/Administration+Applic… == MapSheet == - Map creation: usually, class instances get created directly in the space of the application (see for instance the Diagram application), I would suggest we do the same here if that's ok with you, i.e. directly in the Map space, for maps and for points as well. As for test data: we will need to store test data indeed without the need to recreate it manually everytime, I'm adding an item about that in the "Next steps" section below. - Currently, the sheet issues a call to com.xpn.xwiki.api.Object:getXWikiObject. This method requires programming rights. Most of the classes in package com.xpn.xwiki.api give access to their wrapped core object only to users with programming rights, since it allows to manipulate the data at a low level. I'm adding some links below that may help you understand this aspect (you may know them already). Another option (the one I favour) is to always have the xwiki-platform source code at hand in the IDE, so that one can check what the API does exactly. In an XWiki sheet, we typically don't want the programming right to be required, because sheets are meant to be customized by users, not necessarily by administrators. You will hence have to generate the target JSON without resorting to the wrapped BaseObject. https://www.xwiki.org/xwiki/bin/view/Documentation/DevGuide/Scripting/ http://nexus.xwiki.org/nexus/service/local/repositories/public/archive/org/… - I'm wondering if it's necessary to generate a random id for the map container: couldn't we simply use the lower-case serialized map reference? == CommonMacros == Yes, the "handleMapQuery" macro is in the right direction. == Global code organization == - Currently the code uses one space per class. That's an option indeed, but it seems to me that the usual practice is more to group all classes and sheets directly in the "Code" space. See for instance the blog application: https://github.com/xwiki-contrib/application-blog/tree/master/application-b…. What do you think Caty, all? - We're currently using the "Maps" space. I know the plural was discussed for the app name already, I have no further comment about that. In existing XWiki applications, we use either the singular or the plural for the application space. For instance, the following apps use the singular: application-diagram, application-xpoll, application-meeting, batchimport, application-forum, application-flashmessages (plural in the application name, singular in the space name in this case). Other use plural everywhere: application-invoices, application-ideas. Question to Caty and to all: do we have guidelines for that? This is a minor issue but I have a slight preference for using the singular in the space name, since to me it refers primarily to the application itself, not necessarily to the location where the target instances will be created. Typically, maps could be created anywhere in the wiki. The map app code, though, will be stored in that space. Sorry for being picky, I just want to make sure we all agree on the final name before a potential refactoring becomes more complex. == Next steps == You probably have several of the next steps if not all in mind already, I'm listing some of them below so that we make sure to converge and decide: - Add a livetable listing all existing maps, in the "WebHome" page of the top level application space - Display page titles in the popups - Find a way to plug the query mechanism to the Main.SolrSearch and Main.SolrSearchMacros (so that we can then use facets). This can be a bit tricky, let us know if you need help of course, for this aspect or any other. - Add a full-text input widget to the map that will issue Solr queries asynchronously to the map query service (example: the user inputs "archeologia" -> only pages matching that word will get returned, and only the corresponding points will show up on the map), which will return the documents matching the initial query completed by the added full-text constraint. - Add a contextual paginated list of results. - Add facet filters. - Test data: for now, we have just a few points to test with, but in the near future we will need one or several rich data sets samples to test the application. I'm not sure whether we should put the data and the code to create it in the same repository as the application itself or if we should create a dedicated repository. Typically, I foresee some code that will fetch some data from Wikidata or OpenStreetMap and convert it to several hundreds or thousands of XWiki pages, which in turn will be consumed by a map. Example: the query below executed against the endpoint https://query.wikidata.org/ will return all the museums referenced in Wikidata with their name, their geographical coordinates and their country. We could grab the museum JSON representations and turn them into XWiki pages (I have some code for doing such operations in another context already, I will commit it soon). I'm wondering if we store the test data and code in the same repository (this can amount to hundreds of test pages), or in a distinct one. I'll send a separate question to the list about this aspect. SELECT ?item ?itemLabel ?coord ?lon ?lat ?country ?countryLabel WHERE { ?item wdt:P17 ?country. ?item wdt:P31 wd:Q33506. # is a museum ?item p:P625 ?coordinate. ?coordinate ps:P625 ?coord. ?coordinate psv:P625 ?coordinate_node. ?coordinate_node wikibase:geoLongitude ?lon. ?coordinate_node wikibase:geoLatitude ?lat. SERVICE wikibase:label { bd:serviceParam wikibase:language "en". } } - Regarding paths: I saw you mentioned them on #xwiki, that will be an important feature indeed but I feel like it's more important to make sure that we have a working prototype consisting of a map, a search input, a list and facets with points already, before adding more complex objects to the map, what do you think? Cheers Stéphane Fawad Ali: > Hi Stéphane, Ecaterina and all, > Hope all of you are doing wonderful! > So I am working on the new iteration for the Interactive Maps Application and have made some progress which I would like to share with all of you. new architecture which emphasizes the use of Solr queries for gathering data to display on the map. The space has also been changed to "Maps" as suggested by Ecaterina. > Link: https://github.com/9inpachi/interactive-maps-new > I have not shifted the files to xwiki-contrib because I wanted an initial review if I am doing the right thing so I don't have to change the whole repo twice. > I have been able to generate a map and the application is now able to handle basic queries to include points (markers) inside the map. > I would like you to please give it a look and let me know if I am going in the right direction. :) > Main application pages: > - Maps.Code.Map.MapClass => The main map class > - Maps.Code.Map.MapSheet => Map sheet that renders the map > - Maps.Code.Map.WebHome => Simple form for creating a map object page > - Maps.Code.Point.PointClass => Main class for a simple point (marker) > - Maps.Code.Leaflet => Page where all the javascript and stylesheet extensions exist application > - Maps.MapTesting => Space that contains objects and maps for testing > For adding a Point, create a simple page, add content to it and attach a PointClass object with a value to it. > If you want to have a look at how the map is rendered, see the MapSheet and the relevant jsx (leaflet-main) code on the Leaflet page. > I have attached some screenshots to show what the application does at this point. > Best, > Fawad > On Wed, May 29, 2019 at 5:12 PM Fawad Ali &lt;m.fawaadali98(a)gmail.com <mailto:m.fawaadali98@gmail.com>> wrote: > Stéphane, > Thanks for this. It summarizes our discussions perfectly. :) > It's actually great that we are on the same page now. I will provide you with updates soon. > Best, > Fawad > On Wed, May 29, 2019 at 4:34 PM Stéphane Laurière < slauriere(a)xwiki.com <mailto:slauriere@xwiki.com>> wrote: > Fawad, Caty, all, > A quick update following a video call I had with Fawad: > - The Solr API will be used to query the data to be displayed on the map, and the Solr facets to apply additional filters > - The data returned by the Solr API will be converted to GeoJSON. Later on we may consider creating a Solr service that returns (Geo)JSON directly rather than HTML, to be discussed. > Next steps: > - Creation of a basic Map class with a Solr query field > - Creation of a MapSheet that will render the map itself > - Creation of a Point class that will consist of one field storing the latitude and the longitude separated by a comma (later on, if needed, we will consider storing the latitude and the longitude in two distinct fields) > - Manual creation (or via the BatchImport application) of 1) a set of sample pages that will consist of: a title, some content, and a Point object attached to each page, 2) a Map object with a simple Solr query returning all pages with a Point object in a specific space. > - Conversion of the the results provided by Solr into GeoJSON so that the data can be consumed and displayed by Leaflet > Fawad, please don't hesitate to mention any complementary aspect that I may have forgotten or questions you may have. > Cheers > Stéphane -- Stéphane Laurière XWiki – https://xwiki.com