Current development on JAMWiki is primarily focused on maintenance rather than new features due to a lack of developer availability. If you are interested in working on JAMWiki please join the jamwiki-devel mailing list.

Tech:Maven

ktip.png This page (and all pages in the Tech: namespace) is a developer discussion about a feature that is either proposed for inclusion in JAMWiki or one that has already been implemented. This page is NOT documentation of JAMWiki functionality - for a list of documentation, see Category:JAMWiki.
Status of this feature: IMPLEMENTED. JAMWiki 0.6.0 was the first JAMWiki release to use a fully Maven-ized layout, and JAMWiki 1.0 contained several Maven enhancements.

The old VQWiki repository layout attempted to be compatible with Maven, but full Maven support was never completed. It might make sense to eventually make the JAMWiki repository fully Maven-compliant.


Contents

What is Maven[edit]

Maven is a project management framework. What is meant thereby is essentially three things.

A compilation tool[edit]

Maven is a compilation tool, similar to Makefile or Apache ant.

The central piece of a Maven project is the Project Object Model which describes how the project is build (this can be compared to a build.xml file with conventions)

A set of standards[edit]

But Maven is more than a compilation tool.

In the spirit of "convention over configuration", Maven aims at standardizing a number of things like the build lifecycle or the source layout.

A repository of Java libraries[edit]

Finally, Maven offers a repository of Java libraries that can be downloaded by Maven itself during the different phases of the lifecycle. For instance, Maven can fetch log4j at compile time if the project depends upon this library.

How it can help[edit]

Maven can help:

  • new developers to contribute for JAMWiki has it is very fast to set up an environment: the POM describes everything
  • users to compile the code it they wish to, for the some reason
  • developers to manage the dependencies

What should be done[edit]

  • follow layout convention, in order not to configure it in the POM
  • write the POM, using maven-plugin or calling ant task for specific actions (like generating the parser from the jlex spec files)
  • update the documentation on how to compile Jamwiki

References[edit]

Comments[edit]

Just a note to say that I personally haven't really looked into Maven, but a few people have requested that the project be made Maven-compatible, and unless there is a downside to doing so I'm not opposed to the idea. If someone can lay out any downsides to making this change please do, and it would be nice if someone could provide a checklist of exactly what needs to be changed, but provided there aren't any major disadvantages I'd be in favor of making this change. -- Ryan 11-Jul-2007 12:05 PDT


Thanks for all of your work on this, and please let me know if you need Subversion access (I'd need your Sourceforge ID). Ideally it would be good to get the comments of someone else who knows Maven before committing any changes, but from what little I've read about the Maven project so far it seems like this is a good change to make. -- Ryan 14-Jul-2007 13:15 PDT
My Sourceforge ID is mikegremi. I will only commit pom.xml, mavenize.xml and my plugin. You are right, before moving anything in Subversion, a few people should have tested the new structure. Mike 14-Jul-2007 15:46 PDT
You should now be able to write files to the repository. If it helps you feel free to create your own branch in the repository, although you're welcome to commit small changes directly to the trunk. The only request I have is that commits be small and focused so that they're easy to review - 80% of programming is maintaining code, so it's best to make sure the code that's put in place makes sense to someone other than the comitter!
As to reviewing your proposals, User:RegisDecamps is the one who started this article, so hopefully he'll be able to add his feedback soon. If not I'll make sure to find a few hours to read up on Maven, although ideally I'd like to focus what little development time I have on finishing up the new user permissions code for 0.6.0. -- Ryan 14-Jul-2007 16:04 PDT


Work with Maven[edit]

Test with Maven:

mvn package //creates war file
mvn site //creates website with reports
mvn eclipse:eclipse //updates or creates classpath in eclipse project description. See "Q&A - IDE Plugins"
mvn jetty:run-exploded //runs war file in jetty
mvn javadoc:jar //creates javadoc jar
mvn source:jar //creates jar with sources

To ignore test cases add property "-Dmaven.test.skip"

btw: the jetty thing is only working if the jamwiki-war pom.xml is instrumented with the right plugin, to do that. And if you make the call inside the jamwiki-war directory. what about checking the addition into the svn? -- mbert 12-Aug-2008 01:49 PDT
What change is needed? As long as it doesn't affect non-Jetty users there shouldn't be any harm in committing it. -- Ryan 12-Aug-2008 07:10 PDT

Status[edit]

Provided the two of you are in agreement on the final directory structure and such I've got no objections to the proposed changes, and when you're ready and in agreement you can commit to trunk. As Regis pointed out, please use Subversion's move and rename commands to re-arrange the structure so that we don't lose any revision history - if you're on Windows, Tortoise SVN makes this very, very easy to do, and I suspect Linux has similar tools if you don't want to use the Subversion command-line scripts. I'll keep an eye on things and let you know if I see anything that looks worrisome, but it seems like you both have a good grasp of what needs to be done. In addition, feel free to add yourselves to the CREDITS.txt file - developers are listed alphabetically, and the id in parentheses is the jamwiki.org login.

As a side note, the changes being made for authentication & authorization right now are fairly disruptive, so apologies in advance if I break anything. If you notice problems simply leave a note on Tech:User Permissions. There may be a few more changes to create default roles, which will probably mean you'll need to update your JAMWiki database structure (either via a script or a new install), but I'll try to keep those commits to a minimum and add a note in the commit message when such a change is made. -- Ryan 15-Jul-2007 09:43 PDT

Hi Ryan, whats up with the maven status of jamwiki? I do have very few free time besides my family and the work but I'm familiar with maven. So is there already a branch for it? When I did migrate a projekt to maven 2 I recognized the importance of the recommended directory structure.-- Michael Habbert 31-Jan-2008 00:32 PST
Hi Michael - the project was fully migrated over to Maven by Regis and Mike. For the next release I'm in the process of further modularizing things, so the current Subversion code contains a different layout than what is discussed here. If you've got some experience with Maven and would like to help out it would be great to get some of the Maven reports working again and to hopefully consolidate as much information into the super POM as possible. I'm a novice with Maven, so if there are other areas for improvement then that would be great as well. -- Ryan 31-Jan-2008 07:53 PST


Enclosing (integrating) webtests in jamwiki build?[edit]

Hi Folks.
I'm quite familiar with canoo webtest [1] and interessted to introduce webtests into the maven build. What do you think? The most important question is: what is the benefit for jamwiki? I not familiar enough with jamwiki internals to decide about wheater it is significant or not.
Webtest - configured with xml-files - works like a java-browser you can work with in ant/mvn targets. It will simulate the user/browser interactions with jamwiki. Nearly everything you can do with your browser you can test with webtest-tests (except visual - e.g. 'css' tests). I'm do know much about maven and webtest and so I'm interessted to do the integration (using Jetty for example) to execute the webtests within a special maven target. -- Mbert 20-Mar-2008 01:01 PDT

For my part, I'd be in favor of anything that expands the test coverage provided that there isn't any additional work needed to maintain the code. If canoo can be integrated simply by modifying the pom.xml files and perhaps adding new test configuration information then it sounds like a good value-add. I'll try to take some time to review the canoo documentation over the coming days, but if you're interested in implementing this functionality I'd say go ahead. -- Ryan 20-Mar-2008 07:10 PDT

Hi Ryan, I'm still alive, but lately I was busy. I'm still interesseted in supporting jamwiki with webtest. My personal problem is there are two different tasks to do for me:

  • first: integrate the start of a JSP-Container bevor starting the webtest-suite, calling the webtests from within maven (I found a existing plugin) and the stop of the JSP-Container afterwards. I haven't done this befor!
  • second: creating webtests, a lot of them.

Finally you would be able to compile with all the webtests or without - for short terms. The major problem for me is the first tasks. So for now I'll start with the second one and create some webtest-examples and documentation for the other developers how to use them unless the first-task is resolved completely. The tests will base on a local installed webtest-snapshot and an existing jamwiki installation. (To test the jamwiki installation out of the box on a new system is to complicated for me, to start with - but insteressting ;-)). -- Mbert 29-Jun-2008 07:41 PDT

Hey - glad to see you're back! I'm on vacation for two more weeks so I won't be able to comment much, but since you're mostly changing Maven testing configuration you should be fine to commit just about anything you think is necessary - JAMWiki can always use better test coverage. I'll review more closely when I get back, but thanks for continuing to help out! -- Ryan 29-Jun-2008 12:32 PDT

I will start to document how developers can use the existing webtests. Any wishes are welcome what are interesting or relevent pages and features which should be tested by webtest. Actually I transfered some example webtests to jamwiki.org. To test editing functions we will need a test-user on jamwiki-org (may be a good webtest issue ;-). We'll see -- mbert 29-Jun-2008 12:52 PDT

Using Webtest with Jamwiki

Feedback[edit]

Starting Problems[edit]

Hi Folks, I did call: svn up; mvn package and got:
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to resolve artifact.
Missing:
----------
1) bliki:bliki:jar:1.0-SNAPSHOT
 Try downloading the file manually from the project website.

So is there a special way? I imagine, that you have a version of the bliki.jar in your local repository (~/.m2/repostiory) -- Michael Habbert 01-Feb-2008 06:47 PST

I may have broken things with the changes made during the past couple of weeks - I'll look into this issue this weekend. I didn't think it was necessary to mvn:install bliki, but I'll verify. Thanks for the report. -- Ryan 01-Feb-2008 07:23 PST
Hi Ryan, I examined the source of the problem.
mvn help:effective-pom

brings it to the surface. the repository configuration is not working!

on the multi-modul-level:

   <repository>
     <releases />
     <snapshots />
     <id>org.jamwiki</id>
     <name>Maven Repository for Jamwiki</name>
     <url>file:///<path-to-project>/jamwiki/repository/</url>
   </repository>

but on the jamwiki-core-level:

   <repository>
     <releases />
     <snapshots />
     <id>org.jamwiki</id>
     <name>Maven Repository for Jamwiki</name>
     <url>file:///<path-to-project>/jamwiki/jamwiki-core/repository/</url>
   </repository>

so you see the url property changes from jamwiki/repository to jamwiki/jamwiki-core/repository! And inside the module directories the repository is not found. I'll look for a solution. -- Michael Habbert 01-Feb-2008 08:13 PST

Thanks for investigating - my Maven knowledge is pretty pathetic, so I've been spending a ton of time lately reading Maven tutorials. Any help cleaning up / fixing the current POM is hugely appreciated! -- Ryan 01-Feb-2008 08:27 PST
Hi Ryan, I found a workaround not shure yet about the best solution, but it works for me.

Fix change the url property of the repository from:

<url>file://${basedir}/repository</url>
to: 
<url>file://${basedir}/../repository</url>

so each module who needs the repository will find it on the right place. -- Michael Habbert 01-Feb-2008 10:34 PST

I'm at work now, but I could give this a try later tonight. Alternatively, if you have a Sourceforge ID and want to change this stuff directly just send me your login and I'd be happy to give you Subversion commit access - whichever approach is easier for you. I greatly appreciate the help as I've been fumbling around with Maven lately, and while I'm trying to follow "standard practice" there doesn't always seem to be a clear "right way" of doing things in Maven, so input from people with Maven experience is very much appreciated. -- Ryan 01-Feb-2008 10:48 PST
Ok, mvn clean, package and install is working fine now! But: mvn site is failing - strange!!
mvn clean site

fail's with:

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to resolve artifact.
Missing:
----------
1) org.incava:javadiff:jar:1.0.5

I'm not shure if that is what I expected. But, after calling mvn install (-> installing jamwiki into the local repository) mvn site works as expected. So I beleave this is a personal not understanding of maven and me ;-). Anyway I can check the reports now. -- Michael Habbert 01-Feb-2008 11:16 PST

My understanding is that reports do not inherit - thus each module has no clue that it should check dependencies from the Super POM. There doesn't seem to be a good workaround, but I haven't spent much time investigating. -- Ryan 01-Feb-2008 11:32 PST
Hy Ryan, I did check in one change in the super-pom file and one change in the javadiff pom.xml. While I 'm not an expert on maven 2 I'm interessed in discussing the right way of configuring the maven pom's. What is the usual way you do it inside the jamwiki-project? Discuss the issues on a wiki-page? Ok for me ;-) -- Michael Habbert 02-Feb-2008 02:05 PST
Thanks for the changes. I prefer using the wiki for discussions so that everyone can be involved, and the discussion is preserved, searchable, etc. It's a bit slower than an email list since people aren't immediately notified of new comments, but I think the benefits and lessons learned from using the wiki for discussion are a net win. That said, I've set up a jamwiki-commit mailing list so that anytime code is committed to Subversion an email is sent, which helps alert everyone when new code is committed - you can subscribe from the Sourceforge JAMWiki project page if interested. -- Ryan 02-Feb-2008 09:49 PST


Evolving the reports[edit]

Hi developers, what are the ideas about the reports you want to use and want to take advantage of? I would suggest to get the reports for the whole project, right? If there are some parts we whant to exclude from a special report, we should configure this special report, right? -- Michael Habbert 02-Feb-2008 02:14 PST

I do start to make some tests for my local site (local apache;-) mvn site:deploy -DstagingDirectory=/var/www/jamwiki to see what the reports a like. Everyone with a local webserver on his machine can do the same by adjusting the directory-path. Hi Ryan, do you want to make the site available on the internet in the future? -- Michael Habbert 02-Feb-2008 02:24 PST
Hi Ryan, I would suggest do move the reporting configuration up from the single modules to the super-pom to inherite the configuration to each module and so get a report for each modul, what do you think? -- Michael Habbert 02-Feb-2008 02:51 PST
Hi Ryan, I disagree. As far I see it now there are only two projects: core and web with reporting relevant sources. So I think duplication between core and web will be the best. But at this moment I'm still confused about the javadiff project (still too much to read it all). Right now I think about the site generation and best way to configure it to the needs of all of the developers. I'm right, there is no personal configuration done in the pom right now, right? No profiles defined yet? -- Michael Habbert 02-Feb-2008 07:57 PST
Hi Michael - to answer your questions:
  • It would be nice to eventually show Maven reports on the web site, but it's not critical. Javadocs are currently displayed, and including a few others would probably also be nice.
  • The javadiff project is an external project from http://incava.org/javadiff/. Since he only makes releases as source code, and since there is no Maven repository available, the source code was included in the JAMWiki project. No reports should be needed for that project.
* So what is it with the bliki.jar? It is also an external jar, but provided by some else? For the maven reports there is a problem with handling the two external jars the same way. (My be you find some time to create the maven site on your local machine and see the actual status of the maven site.) The actual status is: the external jar - included from the repository directory will not be documented on the maven jar (as subproject) but the javadiff - subproject is different and treated as subproject and documented on the project site.
-> So you/we have to decide: to compile and assemble the jar external and include both in the repository (so maven site handles them the same way) or accept, that maven will handle the two dependencies different (one as external jar and one as module). So what do you think Ryan? -- Michael Habbert 17-Feb-2008 03:00 PST
Hi Michael - bliki.jar is an alternate parser that was created by User:axelclk. It works similar to the default JAMWiki parser but he maintains the source separately, and I don't believe it is available in an external Maven repository so it was included in its own repository within the JAMWiki project. If there is a better way to handle that file please feel free to change it. With regards to reporting, no reporting should be needed for Bliki and unfortunately the source is maintained elsewhere, so we can't treat it as a standard Maven project. Hopefully that makes sense, but let me know if that's still confusing. -- Ryan 17-Feb-2008 09:14 PST
  • One of my goals during the 0.6.4 release cycle was to make the project more modular, which is why the split between jamwiki-web and jamwiki-core occurred. There have been many requests to allow the JAMWiki parser to be used independently, and I'd like to enable that to hopefully get more people using the parser and thus more people contributing patches and bug reports. Longer term there may be other sub-projects split out from the current projects.
  • In terms of profiles, there is some contributor & developer info defined in the Super POM, but that can definitely be moved around if it's in the wrong place - I was just following examples in a few other project's, which defined contributor/developer at the super POM level.
* Hy Ryan, the profiles are configuration options for different users in the same pom or an external profiles.xml to configure the local maven site directory for example. By using the parameter: DstagingDirectory (see above) I avoided the configuration for now. It might be a good solution for the future to add these profiles to the pom (I would suggest). -- Michael Habbert 17-Feb-2008 03:07 PST
Let me know if I missed anything. -- Ryan 02-Feb-2008 09:49 PST


Problems[edit]

compile failure - tests[edit]

I checked the source and typed:

mvn package 

the compile failed with on test!

Failed tests: 
 testBasicTables7(org.jamwiki.parser.ParseTableTest)
 
Tests run: 218, Failures: 1, Errors: 0, Skipped: 0

I startet the test-method inside the eclipse and it failed again.

Poké Ball
Regular Poké Ball

--------------
Pok&#65533; Ball
Regular Pok&#65533; Ball

is not equal! of coure.

myServer$ locale
LANG=de_DE.UTF-8
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=

So what is it, I have to change? -- Michael Habbert 15-Mar-2008 06:40 PDT

Weird - I don't see that test failure when I build, so it must be environment-specific. For now the easiest solution is to just delete the testBasicTables7 method in jamwiki-core/src/test/java/org/jamwiki/parser/ParseTableTest.java. I'll see if I can figure out why it might produce different output on different systems. -- Ryan 15-Mar-2008 08:28 PDT

As you can see my locale shows only UTF-8 compatible configurations. So I assume the reason of this problem is the file-encoding of the Tables7-file is not UTF-8 kompatible (specially the é). So I changed the file and now the tests is running fine:

Index: jamwiki-core/src/test/resources/data/topics/Table7
===================================================================
--- jamwiki-core/src/test/resources/data/topics/Table7  (Revision 2104)
+++ jamwiki-core/src/test/resources/data/topics/Table7  (Arbeitskopie)
@@ -4,7 +4,7 @@
 !width="225"|Effect
 !width="225"|Games Found In
 |-
-|Pok�Ball || Regular Pok�Ball || All Versions
+|Poké Ball || Regular Poké Ball || All Versions
 |-
-|Great Ball || Better than a Pok�Ball || All Versions
+|Great Ball || Better than a Poké Ball || All Versions
 |}
\ Kein Zeilenvorschub am Ende der Datei

As soon as I have reconfigured my svn-access to sourceforge I will check in the file and someone else have to check. You: Ryan? -- Michael Habbert 18-Mar-2008 06:21 PDT

Thanks! I switched to notepad++ recently, and it seems to have problems creating UTF-8 files, so I'll try to be a bit more careful in the future. Feel free to commit this change as soon as you're able - Sourceforge recently forced all users to change password, so that may explain any SVN problems you're having. -- Ryan 18-Mar-2008 08:03 PDT