Tech:OrphanedPages

This page (and all pages in the Tech: namespace) is a developer discussion about a feature that is either proposed for inclusion in JAMWiki or one that has already been implemented. This page is NOT documentation of JAMWiki functionality - for a list of documentation, see Category:JAMWiki. Status of this feature: IMPLEMENTED. This feature was implemented for JAMWiki 0.7.0 and significantly redesigned for JAMWiki 1.0.

[Edit]Intro

There are multiple ways of implement an Special:OrphanedPages, one of them would query the database direct with the advantage of performance and standard pagination. Here I used only existing interfaces to the WikiData, so only little coding was required to get the page I wanted...

Yes, I know... I should build my own branch in repository and sync it with trunk, but I havn't found the time for taking a closer look to maven/svn. --hp 25-Jul-2008 02:58 PDT

[Edit]The Controller

package org.jamwiki.servlets;

import java.util.Collection;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
import java.util.SortedSet;
import java.util.TreeSet;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.jamwiki.WikiBase;
import org.jamwiki.WikiMessage;
import org.jamwiki.model.Topic;
import org.jamwiki.parser.ParserInput;
import org.jamwiki.parser.ParserOutput;
import org.jamwiki.parser.ParserUtil;
import org.jamwiki.utils.WikiLogger;
import org.jamwiki.utils.WikiUtil;

import org.springframework.web.servlet.ModelAndView;


/**
 * Used for display a list of all orphaned pages in a
 * virtual wiki. 
 */
public class OrphanedPagesServlet extends JAMWikiServlet {

	/** Logger for this class and subclasses. */
	private static final WikiLogger logger = WikiLogger.getLogger(OrphanedPagesServlet.class.getName());

	protected static final String JSP_ORPHANED = "orphaned.jsp";

	/**
	 *
	 */
	protected ModelAndView handleJAMWikiRequest(HttpServletRequest request, HttpServletResponse response, ModelAndView next, WikiPageInfo pageInfo) throws Exception {
		this.viewOrphaned(request, next, pageInfo);
		return next;
	}


	/**
	 *
	 */
	private void viewOrphaned(HttpServletRequest request, ModelAndView next, WikiPageInfo pageInfo) throws Exception {
		Map orphanedPages = new HashMap();
		String virtualWiki = WikiUtil.getVirtualWikiFromURI(request);
		Collection items = WikiBase.getDataHandler().getAllTopicNames(virtualWiki);

		for (Iterator iterator = items.iterator(); iterator.hasNext();) {
			String topicName = (String) iterator.next();
			Topic topic = WikiBase.getDataHandler().lookupTopic(virtualWiki, topicName, true, new Object());
			if (topic.getTopicType() == Topic.TYPE_ARTICLE) {
				ParserInput parserInput = new ParserInput();
				parserInput.setContext(request.getContextPath());
				parserInput.setLocale(request.getLocale());
				parserInput.setWikiUser(ServletUtil.currentUser());
				parserInput.setTopicName(topic.getName());
				parserInput.setUserIpAddress(ServletUtil.getIpAddress(request));
				parserInput.setVirtualWiki(virtualWiki);
				parserInput.setAllowSectionEdit(false);
				ParserOutput parserOutput = new ParserOutput();
				ParserUtil.parse(parserInput, parserOutput, topic.getTopicContent());
				if (parserOutput.getCategories().size() == 0) {
					/*
					 * only mark them orphaned if there is neither category defined in it, 
					 * nor a link to it!
					 */
					Collection results = WikiBase.getSearchEngine().findLinkedTo(virtualWiki, topic.getName());
					if (results.size() == 0) {
						String namespace = getNamespace(topic.getName());
						if (orphanedPages.containsKey(namespace)) {
							((SortedSet)orphanedPages.get(namespace)).add(topic.getName());
						}
						else {
							SortedSet pages = new TreeSet(); 
							pages.add(topic.getName());
							orphanedPages.put(namespace, pages);
						}
					}
				}
			}
		}
		next.addObject("orphaned", orphanedPages);
		pageInfo.setPageTitle(new WikiMessage("orphaned.title"));
		pageInfo.setContentJsp(JSP_ORPHANED);
		pageInfo.setSpecial(true);
	}


	private String getNamespace(String topicName) {
		String[] topicNameParts = topicName.split(":");
		return topicNameParts.length == 1 ? "" : topicNameParts[0];
	}

}

[Edit]The View

<%@ page errorPage="/WEB-INF/jsp/error.jsp" contentType="text/html; charset=utf-8" %>

<%@ include file="page-init.jsp" %>

<style>
<!--
.columnlist { float:left; width:58em; margin:0; padding:0; list-style:none; }
.columnlist li { float:left; width:25em; margin:0; padding:0; } 
-->
</style>

<c:forEach items="${orphaned}" var="ns">
	<h2>:${ns.key}</h2>
	<ol class="columnlist">
	<c:forEach items="${ns.value}" var="item" varStatus="status">
		<li>${status.count}. <jamwiki:link value="${item}" text="${item}" /></li>
	</c:forEach>
	</ol>
	<div style="clear:both"></div>
</c:forEach>

[Edit]The Configuration

In jamwiki-servlet.xml, add the following line to the mapping...

<prop key="/**/Special:OrphanedPages">OrphanedPages</prop>

... and add the Controller:

<bean id="OrphanedPages" class="org.jamwiki.servlets.OrphanedPagesServlet" />

[Edit]Application Resources

[Edit]Restrictions

I think the implementation won't perform on a large wiki. If paging is required, it could be implemented (get all 1000 articles, take the 45 orphaned and paginate them...) but this wouldn't improve performance.

Redirects are treated as links, so an article having a redirect to itself, won't be orphaned, although there isn't any other topic having an link to it. I think this is an matter of taste, for me this is ok.

Thanks for putting this together! As you've said, there are significant performance impliciations, but it may be possible to re-use infrastructure from the "what links here" functionality to display orphaned pages without the need to parse every wiki page... I can take a look, or if you're interested and need pointers let me know and I'll see if I can help out. -- Ryan 25-Jul-2008 07:41 PDT

I did use the functionality from "what links here", the parsing task in the implementation described is for the categories... It is a matter of taste, but articles in categories are linked through the category-pages, so I thought they shouldn't be marked orphaned. If they should also listed as orphaned, the parsing task is obsolete. --hp 26-Jul-2008 12:37 PDT

Thanks for clarifying - I didn't read the code closely enough the first time. I've committed this with some changes as revision 2254. As you've indicated pagination is still a todo, and I think it makes sense to modify existing JAMWiki code to store links and categories in the database - I haven't investigated heavily, but if that data is in the database it should resolve the most serious performance concerns. Thanks again for putting this together, and if you're interested in doing further work let me know and I can set you up with Subversion commit access. -- Ryan 27-Jul-2008 20:57 PDT

Thank you Ryan! I took a look at revision 2254 and I like it! By adding this feature Feature_Requests#Find_all_orphaned_pages_(pages_not_in_a_category) could be removed, I guess. Storing categories in the database would of course make pagination easier and increase performance, but for most instances I'm pretty sure this will be sufficient. --hp 28-Jul-2008 10:04 PDT