Why not do more, much more, with mashups?
Mashups are today used either in widgets supplied by online service providers, or built by hand by developers for particular sites and pages. Thinking about generalizing and simplifying the usage of mashups and the creation of web widgets, came the idea of creating a dedicated online service, Patapage.
Some definitions and references
I’ll start by quoting the Wikipedia definition of mashup:
In web development, a mashup is a web page or application that combines data or functionality from two or more external sources to create a new service. The term mashup implies easy, fast integration, frequently using open APIs and data sources to produce results that were not the original reason for producing the raw source data. An example of a mashup is the use of cartographic data to add location information to real estate data, thereby creating a new and distinct web API that was not originally provided by either source.
And also that of mashup enabler:
In technology, a mashup enabler is a tool for transforming incompatible IT resources into a form that allows them to be easily combined, in order to create a mashup.
Now mashups are mostly used by developers, in particular, JavaScript developers. A nice and typical example is this one:
How to build a personal mashup page with jQuery
and the result is this:
where we see the date collected and displayed. Notice that this is all done purely in JavaScript on the client, also using Yahoo Pipes service; but this requires some development and maintenance, and the server side service has to be already available; you cannot build your own service without installing and maintaining your own server. Another widely used option is to use the web service provider widget (“web widget”), where the style is more or less fixed.
Our idea
Our starting idea was… why not give even more power to site builders on the client side by relying on a unique, simple service, acting both as direct content manager and as “proxy” for other services? The fact that Ajax calls allow to add and call Java scripts on say a button click to an existing, fully loaded page, and then by making cross domain requests it becomes possible to add contents to (over) the calling page, opens a world of possibilities, which up to today has been used only by developers of both client and server side code, by low level access to such features and then lots of hand coding.
Couldn’t it be easier? Couldn’t the HTML designer be spared of (re)building specific client readers and server-side services, and focus on just integration and building a beautiful and friendly site? And even if you are a server-side developer, why not delegating web site content building to a third party, and focus on your main activity?
Why use mashups only to add RSS aggregation, pull data from social sites and multimedia contents? Why not use for any kind of dynamic contents? Why use it in blogs and not web sites? The structure of web site is more complex, and the services needed more refined, at times, but that’s no obstacle to the idea.
Also in case of hand built integrations, the graphic quality of the integration is often not at the same level of the originally designed site.
This is where we started thinking about a new online service, and then started building it up. In mashup terms, we started building friendly data sources to be called and used on button and link clicks; then we standardized even the buttons and links, so that the web builder has just to paste the button script on the page, and gets immediately fully working buttons, links and associated services.
Since we had the service up, almost every day a new possibility of a button comes up, as the sum of serving both the client side JavaScript and the server side content, or proxy to contents, opens a lot of possibilities. See in the side picture just the first suite of buttons.
More are being added, like a Delicious aggregator, Picasa integration, custom designed forms; the existing ones are being extended, like the Wiki-button will allow creations of new pages from links, like Wikipedia.
What superficially looks like a single functionality, like adding a wiki-like button (“PataWiki”), actually adds to your web page a layer of dynamically updatable content, by both the site maintainer and site visitors (if that is what is wanted). This could be a “help” page, contributed contents, a “more in depth” presentation, a “buy now” page (we’ve seen it online used this way already), a way to publish news, …
Each button / link you paste in your pages is associated with a “page id”, which the designer can set by hand: this way the same mashup / widget combination can be used across the entire site, on a single page, or shared across a specific group of pages. This is a flexibility that usual mashup on blogs don’t need, as these are basically single paged, , but is a necessary extension for using mashup effectively on web sites.
-
We also developed a web based configuration of buttons and opened windows, which allows setting of style, dimension and layout. This is what you had to do by writing code and HTML by hand when displaying the contents of mashups – before Patapage 
. Of course if the designer wishes, she can customize the layout of the buttons integrating in the existing layout, as done in this example.
Another recurring problem when adding the possibility of visitors contributing and commenting your pages is that of spam, moderation, e-mail verification and in general of security. Like for example if you enable moderation, you have both to reassure users about their contribution, manage the incoming list to be moderated, notify on approval, etc.: imagine having this by just pasting a little script on your static site instead of configuring a CMS to do that! And you should be able to do this differently for each link or button, if you wish so.
The resulting application is not only a mashup enabler as defined at the beginning, but also a tool providing mashups of all sorts ready to be used.
So we are covering the problems of layout, contents, social contributions, notifications, security… all typical problems traditionally requiring installing your own CMS. This way we are empowering links on the web pages of a site that is preserved as it is, with all its style, effectively progressively making classical CMS’s site development look like stone-age site building.
All this functionality is available in an online service, currently in beta, which we called Patapage1,
from there you can try the demo, see a video, and just enroll and try it on your site. When out of beta, the service will be a paid one (see here for details) which will be a way of keeping it ad-free: as one of our ideals is to give as much as an unobtrusive service as possible, channeling ads would result in just the opposite. We also believe that using a commercial service is a higher guarantee for the users.
I’d be glad to read your impression of this idea – use the comments, or tweet me.
[1] Raymond Queneau has described ‘pataphysics as resting “on the truth of contradictions and exceptions.” Well.. we just liked the sound of the word.
Java is just fine for your online service startup development
There is a commonplace going round according to which Java/J2EE is not an appropriate development platform for startups creating online services. This commonplace is repeated again and again,
e.g. on answers.onstartups.com How to pick a platform for a startup web 2.0 app?
stay away from J2EE. J2EE is too heavyweight. Takes too long to develop. Its great for consultants because it means more billable hours but not good for nimble startups
or
Java is too expensive. Higher initial investment for hardware, human resources, and too much code to write to do just simple thing. A startup can’t afford it.
I hope to show in the following that this is simply false.
The origins of the commonplace
As most commonplaces, they are simply the repetition out of context of something true in context, and all those who have followed Java and its evolution in the last ten years, and the coming and going of many, too many, architecture astronautic – inspired frameworks and fashions, there has been no lack of context where Java has been and is used in absurdly convoluted ways; but this is history, boring, and by now not very interesting. And of course the point is that there is nothing in the nature of the language and tools that forces you to use complex solutions.
My point is the opposite of the commonplace: if you have Java expertise, you are in an advantageous position to start developing an online service.
Using Java for your online service will lead you to a change of focus from corporate development, together with a complete revolution in marketing strategy; it is likely that the business logic involved will be relatively simple, you won’t need to redistribute your code, you won’t need say to certify it for some complex pharmaceutical software standard; other different, complex problems will come to the surface. But this will happen whatever language / framework you are using.
A pragmatic attitude
The quoted common places assume that being a Java programmer simply means the lack of pragmatic sense; but why should that be? Surely there is no entailment there.
Lets tell a few simple facts that most experienced developers are perfectly aware of, but don’t say in conferences: you can write excellent applications and never ever write a single line of unit testing and Javadoc. And put all the time you saved from writing and fixing bugs in the unit tests ( 
) in having human testers try your application, and in this way get feedback worth a million times what your own self-test would tell you (assuming optimistically that the code-based tests are bug free). Also human users will tell you about interface bugs that no code test will ever do, like about the dark gray buttons on a black background, or the unpredictable series of clicks ad back which users find so intuitive and code tests will never cover. You can be self-satisfied with your unit tested and JavaDoc-umented code (remembered to update it all by hand after the last refactorings and fixes?), and release a totally unusable program filled with what for the user look like serious bugs, though you didn’t perceive them as so.
Having a firm pragmatic and user oriented attitude is not at all in contrast with using Java for development: you can be a talented duct-tape programmer, and indeed use Java.
We should all be grateful to the dynamic language communities, that with their repeated successes have shown that the “king is naked” and self-referential practices of formal code quality or blind following of methodologies valid for over-ruled corporations are useless advice in many environments, and that a more socially oriented testing and user interface design is what wins for creating online services. But I believe the development language involved is accidental.
There are all sorts of tricks you can use in Java to avoid complications: as class reloading may cause the web server to drop sessions, you may simply publish some experimental business logic in an included JSP file – yes, as simple as that. Having “duct tape” capacities is not language-dependant, and if on top of your pragmatic skills you use Java, you’ll be perfectly fine.
Note that I am not saying that the development language choice is a crucial step in you startup path (hear the end of this podcast for some wise considerations): I’m just saying that if you have Java expertise, use it, and you’ll be fine.
Online in three months
As a concrete example, working in Java, having a considerable experience in Java web development (we develop Teamwork), and using the simplest solutions and the best tools available, we developed two online services from scratch and put them online in three months, with 2 people working for each service and one shared. The services are these:
http://patapage.com “add dynamics to your web pages by embedding simple widgets”
and
http://bugsvoice.com “turn bugs into opportunities”
and some of the work done for development and marketing is presented here. Anyway in this post the focus is on whether Java helped in the process, or it was a hindrance.
Notice also that the server side development is more and more a fraction of the total development needed; as you can see in the example services above, the design and UI/JavaScript part is getting more and more important.
Java’s comfortable world
If you are a Java developer and are thinking of dropping it, think carefully before doing that: there are many features that you may assume as “obvious” which are not at all available in other environments. A few examples:
Choices. In the Java world we are happily used to having choices; that is, for a well defined common problem, not only you often find a solution, you most often find more than one. This is in stark contrast with “canned” environments.
Refactoring. You take as granted Java’s clear syntax, and all the advantages of its static typing. If you are fascinated by dynamic injection, consider for example that you’d simply have to drop your trust in refactorings to work, which with Java (and Intellij ) is close to certainty. And this may mean a shift of focus from the core of your problems to… find-and-replace by hand, a reversion to stone age code writing – sounds silly enough. Java core is stable, and basic signatures can’t be overwritten: in many case, this is a quality. Listen to this very interesting Java Posse podcast for competent discussion of Static vs. Dynamic Typing, which will help you get a clear picture about the myths of dynamic language productivity.
The largest server side development community you can get. That’s a simple fact. See this and this, for example.
Good producer support. Server side Java really runs well everywhere, and also is fast everywhere (see e.g. Twitter: Service vs. Platform), and keeps getting faster; with today’s servers that is not that relevant. Anyway, it feels good to know. Being fast, no great hardware support is required, which is the opposite of the commonplace quoted in the beginning.
If you are using a relational database, all the components scale, with well known and documented practices.
A real virtual machine. Virtual machines should run on all main OS out there, and with identical functional coverage. Otherwise its cheating.
All this makes Java a great platform for almost any kind of development; but lets see some specific needs for online services.
Some positive examples
Lets see how the Java platform, available APIs and some tools developed by us along the way can help you in several concrete cases.
While developing your new online service you will probably need to meet these or similar problems along the way, which you will hardly have met doing say intranet corporate development:
- Cross-site scripting
See a definition here. Examining these kinds of problems, we put together a quite complete Java HTML sanitizer here, which everybody can freely use. The development process is described here.
- Cross-site request forgery
See a definition here.
- Filtering spam
For this there are many solutions, see this one for example:
http://www.theserverside.com/tt/articles/article.tss?l=UsingCI-Bayes
We are experimenting using an online service; the nice thing about using Java is that because it is so widespread, everybody provides the Java stub for their services, ready to use.
- Exposing a (RESTful?) API
Here our solution is under development (but almost there), in our experiments we are using JSON-lib.
- Configuring/scripting your service
A (great) example where we used the power and openness of Java to “talk” with a wider audience is in BugsVoice rule scripting language, which is simply JavaScript; given the large and cross server-platform competence in JavaScript, it is the ideal language for letting a wide audience script your application. See here for details.
- Integrating Open Id
We use openid4java, but of course here too there are plenty of choices.
- Talking with Google applications, Twitter, … .
Here too the fact that everybody is exposing stubs for Java is just great, and saves a lot of time.
- Full-text and even smarter searching
Lucene here is the framework to mention. Actually there are many cases, in particular in online services, where full-text search is not enough; never tried Google? So we did some work on this theme too. Here is a quote from Smarter search and recent object functionality :
Here we examine a technique to improve usability in complex applications by introducing smarter search and “recent objects” functionalities. As usability becomes more and more a crucial feature of applications, helping users with full-text search and recent object lists may still prove insufficient. You may need to go beyond these features, by having a way to keep track of “most used” objects, which will help to:
- guess what you are looking for
- find what you are searching for
(Some links on this problem:
Google’ page rank paper: The Anatomy of a Large-Scale Hypertextual Web Search Engine
A discussion on badges: http://stackoverflow.com/questions/135647/how-do-badges-work-in-stackoverflow
An introduction to full text search: http://www.javaworld.com/javaworld/jw-09-2006/jw-0925-lucene.html
Hibernate full-text search: http://www.hibernate.org/410.html
Our contribution to Hibernate full-text search: http://www.hibernate.org/432.html)
Conclusions
The basic question is: does the Java environment help solving your real problems, apart from any architecture astronautic fad? I hope that the examples above show that it does.
Introducing Patapage
We just put online a new web service, Patapage:
Patapage is a way of adding services to web sites (also static ones) in a most simple way: just by adding buttons. There are buttons for socially contributed contents, like adding a wiki-like layer to your pages, and also comments, image galleries, contact forms, rating, retweet, Google or Twitter searches, windows on other websites, annotated feedbacks, and so on. We are adding more services almost daily.
By relying on this service, you are avoiding any kind of server maintenance, and getting some nicely designed and customizable additional layers on your site. All this in an unobtrusive, simple way.
Anyone who is familiar with adding Google analytics or Twitter gadgets on their site will find Patapage simple to use and in line with the current evolution of the web.
There is simply nothing of the sort, and to see the demo you don’t even need to enroll: so just give it a try!
4 comments