Repository navigation
Support for XSD in version 1.1 #222
Description
Activity
first of all thank you for this excellent VSCode extension.
Thanks!
@jtschea we have a PR which supports XSD 1.1 in eclipse-lemminx/lemminx#515 but it has been not merged because
- it will grow up the size of the vscode-xml because it requires adds of (non standard) Xerces JAR which supports XSD 1.1
- we found a lot of problems that we fixed but we are not sure that it will cause some regression.
Am also looking for XSD 1.1 support, as I have XSDs for config files that require the expanded
xs:allrules (specifically allowing unbounded occurances)Reacted by glenveegeeHi Angelo,
thank you very much for answering.
I was finally able to try out your PR within vscode-xml but still get errors when using XSD 1.1. I cloned and built your branch
xsd_1.1and copied the resulting jar toextensions/redhat.vscode-xml-0.10.1/server. The jar was extracted to the same location as well.Is this the correct way to experience XSD 1.1 in VSCode or am I doing something wrong?
Thanks in advance and best regards,
JuppHi Everyone!
I am also looking for XSD 1.1 support, especially for xsd:assert and xsd:assertionI was able to build the xsd_1.1 branch (https://github.com/angelozerr/lsp4xml/tree/xsd_1.1), but unfortunately assertions do not work properly there. The reason is wrong version of 'org.eclipse.wst.xml.xpath2.processor' lib in the pom.xml. Xerces is built against version 1.2.0 of this lib, however in pom.xml 2.1.100 is used. Version 1.2.0 is not published to maven repo (at least I was not able to find it).
I thought it will be a good idea to make an uberjar of xercesImpl.jar, including xpath processor and icu4j bundled with xerces distribution binary.
I already did this locally and so far it works well. Use of bundled icu4j (which has only 'normalizer' module) also solves the size problem of lsp4xml uberjar. I will publish everything I have soon.@jtschea Try adding this into your settings.json:
"xml.validation.schemaVersion": "1.1"
This turns on 'XML Schema 1.1' mode for the validator.Reacted by Shlomo Gottlieb, Andrew Chin and glenveegeeHi @alex-bel thank a lot for your feedback!
You mean https://github.com/angelozerr/lsp4xml/blob/xsd_1.1/org.eclipse.lsp4xml/pom.xml#L103 is not enough? Please create a PR to fix it.
I thought it will be a good idea to make an uberjar of xercesImpl.jar, including xpath processor and icu4j bundled with xerces distribution binary.
We create an uberjar for LSP4XML which includes xerces, is it not working?
@jtschea Try adding this into your settings.json:
"xml.validation.schemaVersion": "1.1"Indeed, @jtschea please add this settings.
@alex-bel is there any chance that you create a PR from my branch https://github.com/angelozerr/lsp4xml/tree/xsd_1.1
Please note LSP4XML should be move too soon to Eclipse.org. There will have some packages rename.
Hi @angelozerr
I will try to give more detailed explanation.
Let's say you want to use Xerces4j with XML Schema 1.1 support. First problem you face immediately is that you cannot find official (published by developers) artifact in public maven repositories.
However, you have official binary distributions (available here: https://archive.apache.org/dist/xerces/j/binaries/ ). If you download Xerces-J-bin.2.12.1-xml-schema-1.1.zip and unpack it you'll see that it contains not only Xerces itself (XercesImpl.jar), but also all its dependencies, including transitive ones.Some folks already uploaded xercesImpl into Maven central (https://search.maven.org/search?q=xerces ). Some of them ( https://search.maven.org/artifact/com.rackspace.apache/xerces2-xsd11/2.11.2/jar ) added reference to version 2.1.100 of org.eclipse.wst.xml.xpath2.processor:xpath2, which breaks xsd:assert support. Other guys added reference to correct version 1.2.0 (https://search.maven.org/artifact/org.exist-db.thirdparty.xerces/xercesImpl/2.12.1/jar), but the artifact itself is not available in the maven central.
Long story short, you cannot assemble working version of Xerces with fully operational Schema 1.1 support from what you can find in Maven central right now.
Therefore, you have to make and upload your own artifact(s). You have 2 approaches: either upload XercesImpl and its dependencies separately, or create and upload an uberjar (that consists of xercesImpl.jar, org.eclipse.wst.xml.xpath2.processor_1.2.0.jar and optionally icu4j.jar).I took the second way. There is ( https://github.com/alex-bel/xerces4j-with-deps) a simple gradle project that downloads official Xerces distribution, creates uberjar and uploads it there: https://dl.bintray.com/alex-bel/alexbel-repo/xerces4j-with-deps/xerces-schema-1_1-icu/2.12.1.1/
There is my version of xsd_1.1 branch (https://github.com/alex-bel/lsp4xml/tree/xsd_1.1) that uses xerces-schema-1_1-icu:2.12.1.1 artifact. I have also rebased it onto master to fetch recent changes. You probably don't want to merge these changes into your repo since pom.xml contains reference to my maven repo in bintray.
My suggestion is that you make and publish your version of Xerces under your groupId into maven central, since you already have access to it. If you choose to publish uberjar, you can use my publisher. If you decide to publish xercesImpl and its dependencies as separate artifacts, I also can assist you with making of gradle script.
With that's being said, I can answer your questions
You mean https://github.com/angelozerr/lsp4xml/blob/xsd_1.1/org.eclipse.lsp4xml/pom.xml#L103 is not enough? Please create a PR to fix it.
Reference to xercesImpl is OK. Version of org.eclipse.wst.xml.xpath2.processor:xpath2 is the main problem (https://github.com/angelozerr/lsp4xml/blob/xsd_1.1/org.eclipse.lsp4xml/pom.xml#L112). I cannot make a PR since version 1.2.0 of org.eclipse.wst.xml.xpath2.processor:xpath2 is not uploaded into maven central.
We create an uberjar for LSP4XML which includes xerces, is it not working?
I was referencing to "xerces-with-dependencies" uberjar, which consists of xercesImpl.jar, org.eclipse.wst.xml.xpath2.processor_1.2.0.jar and lightweight version of icu4j.jar
Reacted by phlppnhllngrHello,
Can you tell me if the support of XSD 1.1 will be implemented sooner or later?
I need that especially for xs:assert... :)
Thanks for your useful tool!Reacted by glenveegeeCan you tell me if the support of XSD 1.1 will be implemented sooner or later?
For the moment it's not in our priority and it requires to use a non official Xerces distribution. I find it's annoying.
@fbricon have you some idea about that?
Thanks for your useful tool!
Glad vscode-xml please you. Thanks!
For what it's worth, I'm also looking for VSCode extension support for XSD 1.1, especially to be able to add xs:assert elements.
Anyway, thanks for the great tool and I hope you guys will find the time to support it :).
Reacted by DeadRabbits307 and TLWould also like to see XSD 1.1 support in VS Code for the xs:alternative element.
Reacted by TLHello everyone. Is this feature still planned? Love the extension but since I'm working with XSD 1.1, the validations won't work for me. Thanks!
While Xerces will not provide an official support with XSD 1.1, we cannot provide this support.
Ok, thanks for the update!
While Xerces will not provide an official support with XSD 1.1, we cannot provide this support.
But what about https://xerces.apache.org/xerces2-j/releases.html ? Xerces-J 2.12.2 has many bug fixes and xsd 1.1 seems less experimental, more 'supported'? @angelozerr I am keen to see this in Eclipse too, idk why its not there yet, as so keen to use xs:assert and dont want to go down the schematron route or leave ide for this task. Anyhow, I came here from eclipse-lemminx/lemminx#363 so maybe 2.12.2 update can fix both?
We ready use Xerces 2.12.2 version, but when I try to create te XSD 1.1 (according the the sample https://xerces.apache.org/xerces2-j/faq-xs.html):
package org.eclipse.lemminx; import javax.xml.validation.SchemaFactory; public class Test { public static void main(String[] args) throws Exception { SchemaFactory sf10 = SchemaFactory.newInstance( "http://www.w3.org/XML/XMLSchema/v1.0"); SchemaFactory sf11 = SchemaFactory.newInstance( "http://www.w3.org/XML/XMLSchema/v1.1"); } }
I have the following error:
Exception in thread "main" java.lang.IllegalArgumentException: No SchemaFactory that implements the schema language specified by: http://www.w3.org/XML/XMLSchema/v1.1 could be loaded at java.xml/javax.xml.validation.SchemaFactory.newInstance(SchemaFactory.java:247) at org.eclipse.lemminx.Test.main(Test.java:13)Not sure standard Xerces 2.12.2 provides the XSD 11 support?
I am not very familiar with Xerces, but when I look at the download page I see that there are two downloads for Xerces 2.12.2, and only one of them indicates that it supports XSD 1.1:
And when I download the Xerces-J-bin.2.12.2-xml-schema-1.1.zip version, I am able to create a new
http://www.w3.org/XML/XMLSchema/v1.1instance without error. I haven't actually tested the XSD 1.1 feature that I'm looking for, but maybe this gives some clues about what version of Xerces actually has v1.1 support.Reacted by Dino Chiesa@eminence I remember now the problem. Xerces xsd 1.1 is not published on maven central which is very annoying to build lemminx.
More I wonder if xsd 1.1 is mature compare to xsd 1.0, why xerces maintains 2 branches of xerces?
@angelozerr Ahh, I see, indeed.
I found this Xerces JIRA task that mentions publishing the Schema 1.1 version to Maven: https://issues.apache.org/jira/browse/XERCESJ-1736
@angelozerr Ahh, I see, indeed.
I found this Xerces JIRA task that mentions publishing the Schema 1.1 version to Maven: https://issues.apache.org/jira/browse/XERCESJ-1736
Thanks. I added a comment to that ticket so fingers-crossed we get a mvn central xsd 1.1 library soon.
Reacted by Andrew Chin, Eragon-Brisingr, DeadRabbits307 and Michele LocatiSo is this a dead issue? I'd like to be able to write 1.1 schemas to validate some pre-existing data in vscode. I'm willing to do a bit of building locally as well to get it done, but I don't see clear instruction here and the linked branches seem very out of date.
Are extra modifications to the extension source necessary to get 1.1 working, or just switching out the Xerces dependency jar? Is there a reason someone can't just make a branch and manually check in the correct Xerces jar? It seems to be the fastest way around the Mavenless mess Xerces xsd 1.1 support has found itself in. Would also be nice if there was some kind of feature toggle, so I could just download the extension in vscode to get 1.0 support, and download the Xerces jar and point a setting at the path to get 1.1 support.
So is this a dead issue?
No, just wait for Xerces provides their XSD 1.1 as an official maven dependency. @mrglavas
have you some idea about this XSD 1.1 support in Xerces?Are extra modifications to the extension source necessary to get 1.1 working, or just switching out the Xerces dependency jar?
Badly it requires some code changes (see my old PR)
Is there a reason someone can't just make a branch and manually check in the correct Xerces jar?
We want to use an official Xerces dependency and not a build that we could do it. If Xerces doesn't provide this XSD 1.1, I wonder if Xerces team think the support is robust? We have no time to fix / improve this Xerces XSD 1.1 support.
So my understanding of the issue so far:
- RedHat XML currently only supports xsd 1.0. This is determined by its language server, lemminx
- lemminx in turn only supports xsd 1.0 because it depends on Xerces to handle the xsd / schema validation. Xerces publishes multiple versions of their jar, wherein the default one only supports 1.0 and another one supports 1.1.
- lemminx wants to only depend on artifacts published to Maven Central, but word on the street is that Xerces will not do that (at least they haven't for many years, and this was posted over half a year ago):
If so, I've contacted the Xerces-J maintainers, and they answered that they won't publish on Maven, but can accept volunteers that do that.
In other words, there doesn't seem to be an easy path forward for people like me. But I see a few things that could make it possible:
- The code changes branch with experimental XSD 1.1 support could be rebased on main / updated. It doesn't have to get to main, just be in a state where someone could get the Xerces xsd1.1 jar themselves and build locally. As-is, the lemminx xsd_1.1 branch is 661 commits behind main and I expect there's a lot of conflicts I don't have the knowledge to resolve.
- The lemminx xsd_1.1 support could be merged to main but blocked behind a feature toggle. Users would only be able to access the 1.1 support if they had Xerces jar downloaded locally and passed the path through some configuration / command-line parameter. This would only work for lemminx for Java, not native binaries.
My point is this is frustratingly close to being usable, my main worry is that the lemminx branch is too out-of-date to work with the extension. If I tinker with it more and get something working I'll post a guide here, but I'd really appreciate if the lemminx branch could be updated or merged to main behind a feature toggle.
Edit: it looks like someone had the idea of building locally before me and they kindly posted links to their scripts/the process. Something is still broken for me and I'm not dedicated enough to this to push much farther. My process so far:
# clone lemminx, get the xsd_1.1 branch git clone https://github.com/eclipse/lemminx.git cd lemminx git fetch origin xsd_1.1 git checkout xsd_1.1 # download mlocati's xerces install script curl https://raw.githubusercontent.com/concretecms/concrete-cif/0a35b5d08792ebca58ea7065d720d25c88d2058f/bin/install-xerces-m2.sh > install_xerces.sh # run the script, this should install xerces 2.12.2 with xsd 1.1 support to the ~/.m2 folder ./install_xerces.sh # update the dependencies
I updated the dependencies by opening lemminx/org.eclipse.lsp4xml/pom.xml and updating xercesImpl to version 2.12.2 (line 106).
mvn clean package
Results in:
[ERROR] Failed to execute goal org.apache.felix:maven-bundle-plugin:4.1.0:manifest (bundle-manifest) on project org.eclipse.lsp4xml: Execution bundle-manifest of goal org.apache.felix:maven-bundle-plugin:4.1.0:manifest failed.: ConcurrentModificationException -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.felix:maven-bundle-plugin:4.1.0:manifest (bundle-manifest) on project org.eclipse.lsp4xml: Execution bundle-manifest of goal org.apache.felix:maven-bundle-plugin:4.1.0:manifest failed.I can't say I'm terribly familiar with Maven as I don't work on Java build systems often, so not sure where to proceed from there. Hopefully someone else has seen this kind of issue before and can pave the rest of the way.
The PR is too old to use it. We need to re-create the PR.
The main problem is that vscode-xml embeds in a oneJAR lemminx + xerces, so I fear that it will hard to use an another JARs.
It is not a trivial issue and badly our team is busy with another projects.
So is this a dead issue?
No, just wait for Xerces provides their XSD 1.1 as an official maven dependency. @mrglavas have you some idea about this XSD 1.1 support in Xerces?
Xerces' relationship with Maven is complicated. Maven has never been part of Xerces release process. Xerces is built with Ant and the developers never took an interest in anything about Maven. Other people from the Apache community have been responsible for uploading releases. So there has been a distribution (https://xerces.apache.org/mirrors.cgi#binary) of Xerces with XSD 1.1 support for many years, it is stable and complete, but it's possible no one ever publishes that to a Maven repository, perhaps because whomever has been publishing Xerces to Maven doesn't use it. I always challenged people asking for this stuff to get involved in the project and become committers. When they don't volunteer after making that request, nothing happens.
I always challenged people asking for this stuff to get involved in the project and become committers. When they don't volunteer after making that request, nothing happens.
Going to be honest here, I'm not your guy. I don't really work in Java / JVM languages outside work and mostly work in Python. I don't have experience getting things into Maven Central and don't really plan to get any. IntelliJ's quick guide to getting things published seems like it involves getting special accounts/permissions and uses signing tools I don' have on hand. I'm already involved in OSS projects that are taking up all my time, I was just looking for a quick-to-install tool others on my team can use to get autcomplete and red squiggles in certain XMLs. If it's this much trouble then I think I have to call it here, because even if I get it working for myself I don't think the rest of my team will be willing to jump through the same hoops.
Thanks for the quick responses, though, and thanks for the extension as it is! I'm able to make-do with xsd 1.0 for most of what I want to accomplish.
Thanks for the quick responses, though, and thanks for the extension as it is!
Glad vscode-xml pleases you! Thanks for this kind of feedback!
Supporting XSD 1.1 (for Red Hat) is not our priority, but we are totally opened to support it if we have some contributions.
@mrglavas as you are using LemMinx in your Liberty project, I wonder if you could be interest with XSD 1.1 support?
@mrglavas as you are using LemMinx in your Liberty project, I wonder if you could be interest with XSD 1.1 support?
@angelozerr Yes, we use LemMinx through: https://github.com/OpenLiberty/liberty-language-server in the Liberty Tools plugins/extensions for Eclipse, VS Code and IntelliJ. I'm not aware of a requirement to support XSD 1.1. Our tools generate / utilize a schema for the Liberty
server.xmlconfiguration but would expect that XML schema is XSD 1.0.One of my colleagues was a member of the W3C XML Schema working group which produced the XML Schema 1.1 standard. We were very active in its development and Xerces' implementation was used to help provide feedback and validate that the spec could be implemented successfully. I know folks have been concerned about Maven but I actually think a major reason for slow adoption has been that XSD 1.1 support never made it into the JDK. The spec took 8 years to mature and I think by 2012 it had missed the opportunity to be included in JAXP. I was a member of the JAXP expert group in its later years and XSD 1.1 was never in scope. Perhaps people who want to see XSD 1.1 support should approach the OpenJDK project: https://openjdk.org/ with their interests and maybe one day Java will include it.
Reacted by Nirmal GuruI always challenged people asking for this stuff to get involved in the project and become committers. When they don't volunteer after making that request, nothing happens.
@mrglavas as you mentioned here above and there that you “keep encouraging the community to contribute to the project”, please consider answering the person who is trying to Mavenize the project and asks for some information there (unless I missed something and it has been done already of course).
Dear developers,
first of all thank you for this excellent VSCode extension.
I need some advanced XSD features that are only available in version 1.1. My question is, if LSP4XML supports this feature.
The XSD indicates the usage of XSD 1.1 like this:
I am using a "xs:group" as a child of "xs:all" which is allowed in 1.1 but I am getting an error.
Thanks in advance and best regards,
Jupp