Hi Everyone,
The purpose of this post is to outline the current iDempiere state of the union (2016-05-06) as is observed by me and many of the ERP Academy members.
iDempiere officially started in April 2011 as a fork of ADempiere. There were two purposes in starting the project (1) enable ADempiere to use OSGi to create a much more extensible application architecture and (2) resolve a conflict between the ADempiere community and Carlos / Heng Sin that allowed Carlos and Heng Sin to continue contributing to the code base. Both purposes were achieved.
OSGi and Modularity
Between 2011 and 2013 (inclusive), Carlos (1300’ish) and Heng Sin (1300’ish) contributed to over 2600 commits to the iDempiere code base. There was much work to be done to learn OSGi, restructure the project to break it down into modules within a single repository, and create interfaces so that other developers can create OSGi plugins to extend iDempiere without modifying iDempiere’s core. ADempiere was one monolithic Eclipse project. iDempiere is now a collection of Eclipse projects working together to form a single application. As a result, there are over 30 API interfaces available today to support non-core plugins. Below is a list of all OSGi interfaces created to date. Many of these interfaces have tutorials to help developers extend iDempiere without modifying the core application.
User Interface
Probably the second biggest iDempiere development effort was updating the user interface (overall look, feel and navigation). Gone are the indented side-tabs of ADempiere. iDempiere uses a tab architecture that resembles that of Openbravo, another Compiere fork/clone. There was some concern in the beginning over the change; however, even many of the old ADempiere guys are starting to see the utility in the user interface redesign. I personally like the change. I believe it is generally accepted as an improvement over the old ADempiere UI. iDempiere’s interface is clean, and it easy to navigate for new users.
Tucked away in the new user interface are many small improvements over ADempiere. As a 14-year veteran of Compiere/ADempiere and now iDempiere, I cannot tell you how important the little changes are to the overall stability and usability of the system. Mouse-less navigation, quick info widgets, in-window graphing are just a few of the improvements that make iDempiere more enjoyable than its predecessor. Here is a list of all new documented features.
Performance
One of the challenges of an open source application is maintaining proper use of best practices throughout the application. This challenge is exaggerated when the application is an ERP containing over 4K classes, over 800 tables and over 150 views. One of the major undertakings of the iDempiere development team was to review and update the code to find memory and database transaction related leaks. The fix was to update the problem areas to incorporate best practices. The results is a much smaller and more stable iDempiere. ADempiere suffered from daily to weekly reboots under heavy load. The ADempiere server size required to maintain a group of users was shockingly high to me. iDempiere runs on smaller hardware for a greater amount of time. By my estimation, the average user session memory footprint is about 300MB per user. This number varies depending how the disparity of user types (accounting vs order entry vs fulfillment); however, the number is amazingly steady over time – meaning that it does not grow due to memory leaks.
Application Interface
Along with the improved human interface, iDempiere gained ground with CSV and web services improvements. You can now insert, merge and update data into almost any window from a csv file. The csv importer can de-reference foreign keys by search key or name. It can import data into parent and child tables from a single import. It even simulates a human importing the data. Said another way, the column order in your csv file matters because the importer acts as if a human is entering the data field-by-field. This means that callouts and other UI related tools are executed.
Web services in iDempiere received an upgrade. The two biggest improvements are (1) the addition of composite methods in a single call using the CXF framework and (2) improved performance by caching session details between calls. When most people think of web services, they think of REST. When people find out that iDempiere’s Web services are mostly used via SOAP they either raise their eyebrows in surprise or lower them in concern. While you can interact with iDempiere using an XML REST interface, it is more common to use iDempiere’s SOAP interface because of its composite nature. Composite means you can embed multiple calls in a single SOAP envelope. What used to take 10+ REST calls to create an order can now be down in a single SOAP call. The reference look-ups to ID’s for business partners, locations, products, services, etc… can be embedded in a single call. That same call can include both the order header and order lines as well. The result is fast and simple communication.
Environment Migration
Maintaining a database-centric application across many developers is challenging. It is even more challenging when the Application Dictionary is maintained in the database like is the case for iDempiere. This is true for the core iDempiere development team as well as any larger company who uses iDempiere as its ERP. iDempiere does a great job of supporting large development teams by providing strong migration tools. Let’s assume you are a $400M company using iDempiere. Let’s assume you have two external teams that perform enhancements for you. Assume you have an internal team performing quality assurance. Assume you have a formal acceptance and release process.
All of these assumptions mean that you need to be able to orchestrate a release from individual enhancements and propagate that release through multiple environments including developer testing, quality assurance, user acceptance testing and finally production. At any stage a given feature may fail, but it should not stop the entire release. iDempiere gives you tools to integrate migration scripts with source code control to ensure you can manage a complex release cycle.
Revitalized Community
When I created the ERP Academy, iDempiere did not exist as a viable project. It was still in its infancy. One of my concerns about building my business around ADempiere was that it seems to be aging, and there was no real effort to keep the application current. This was evidenced by the application structure, Java version, Tomcat version, etc… iDempiere (Carlos and Heng Sin) seems to have revitalized the community. This was accomplished with the previously mentioned section as well as an overall technology upgrade. The result is a vibrant developer community.
The Results
The result is that a $20M to $200M USD multi-national, multi-currency business can successfully go live with a nicely-featured ERP with less than $10K USD external expenses if you self-install using the ERP Academy. The same company can go live using a professional services company for an expected external expense of $80K USD. If you use both professional services and the ERP Academy, your external expected costs land in-between these numbers. When you compare these expected costs to quoted installations from SAP, Info, Dynamics, Sage, etc… the cost difference can be quite striking.
These numbers make sense because there are no licensing costs, no ~20% annual software maintenance cost, and the bulk of your external expenses come from (1) learning and evaluating the application, (2) importing data (3) configuration, (4) print formats and reporting, and (5) automating manual processes. You either pay someone else to perform these tasks for you or you use internal resources.
Nothing is Perfect
Up until now we have discussed what the iDempiere community has accomplished, which is quite impressive. There is no doubt that iDempiere is a better application in terms of its user interface, stability and developer tools (collectively called ‘the framework); however, one question I have is: “is iDempiere a better ERP”? When you look at all the new features, all the development tickets, and all the commits (of which there are many), what new business feature were created? What can an accountant or an warehouse manager do the he/she could not do in 2011?
By the Numbers
To answer this question, I did some research.
How many changes are made to what classes? I believe this is a good proxy for where developers are spending their time. Below is a list of the top 50 most committed/changed files in iDempiere since Jan 1 of 2014. These files have a similar order when you go further back to 2013. I choose 2014 to prevent the OSGi restructure from skewing the results.
Out of the top 30 files changed, only four are business logic classes (MInOut, MOrder, MPayment and MInvoice). The rest relate to the application framework. Then the question becomes, what types of changes where made to these business logic classes? Were they simply bug fixes or was new functionality added? Below is the list of changes to MOrder.
Only two tickets gave business users the ability to do something they could not do otherwise: 2668 – adds a checkbox to a locator to exclude from demand operations and 1999 – allows over-shipping product. I performed a similar analysis of MInOut and found only one additional feature created by Deepak and the Logilite team (IDEMPIERE-1770) that allows you to manually add attribute set instances to shipments and receipts. This means that out of over 600 commits to the top 13 most active files, only four business logic features were added. The balance went to support the previously mentioned framework improvements.
This is not to say there are no new features in iDempiere. It is quite the opposite. The iDempiere application framework has had many, many new features/improvements. There are also some business features in less prominent classes. Country Group is an example. The challenge is finding business enhancements in the sea of technical/application features (see the last two prominent iDempiere branches 2.1 and 3.x). What is missing is business logic improvements.
Is Lack of New Business Features a Problem
There is no question that ADempiere’s application framework needed improving. The iDempiere’s focus on this one goal is commendable. I am grateful for the efforts. My life is better and easier because of the above results. However, for our community to thrive, we need to focus on solving business problems as well. Just because iDempiere can scale to hundreds of concurrent users does not mean the users of these large systems can do their jobs. To answer the above question: yes – the lack of new features is a problem.
This question became painfully obvious to me when a former SAP integrator can to me with an Invoice Price Variance improvement. The situation was clearly outlined. The documentation included how SAP performed the transaction. When we approached Carlos, he turned down the work. The response was there there was no time. At some level this makes sense – he is the steward of iDempiere. He has much to do, and he and his family are practically the only people who can directly commit to iDempiere.
Is it Time for a New Focus
Given that the development team has been so successful at improving the iDempiere framework over the last four years, is the answer a matter of changing focus? Do we just point our efforts to creating new business features? One might argue yes. However, I do not believe this is the correct answer.
ERP is an amalgamation of many many topics. Business topics include accounting, order management, invoicing, cash management, inventory, sales, purchasing, project management, etc… Technical topics include persistence, object life-cycle, messaging, cache management, user interface, web services, API design, OSGi. Any one of these items (example: accounting) is an area of study and expertise. The complexity of large systems is far beyond an one person’s grasp. This is especially true when you realize that maintaining software is more than just creating new features. These features along with the many features that already exist need documentation, testing, and maintenance.
In an ideal world if you want a new tax feature, you get in the tax feature development line. If you want an invoicing feature, you get in the invoice feature line. If you want a new accounting feature, you get in the accounting feature line. This paragraph could go on and on and on across all ERP topics listed above. The point is that if you have a tax feature, the tax development expert, who focuses only on taxes, would be able to quickly understand your proposal and map it into the existing software in a meaningful and consistent way. What is troubling about iDempiere is that there is only one line for all topics.
That is not entirely true. There is actually 1.25 lines. There was an eye opening moment in last year’s iDempiere conference when Thomas Bayen, employer of Carlos’ son, claimed to the entire conference that he had no issues getting code contributed to the core. The inference was that he had a fast track to contributing enhancements. I do not think that Thomas meant any harm or disrespect; however, his words spoke to a bigger issue, and it struck a sour chord with many of the attendees.
Modularity and Multiple Functional Lines
Back to the question of “is the answer simply a matter of changing focus to business features”… I believe there is one last major framework enhancement needed to help iDempiere grow to the next level. I believe the single best technical/application enhancement that Carlos and the team could do to improve iDempiere is to break it into its functional pieces across multiple repositories. This way the tax experts can maintain the tax bundles. The inventory experts could maintain the storage bundles. The accounting experts can maintain the accounting bundles. The UI experts, the persistence experts, the web services experts, etc… can all execute within their respective disciplines.
This concept is especially important for classes like Orders and Invoices. In today’s world, there is a spaghetti connection between Orders, Invoices and Payments. The alternative is to separate the repositories by functional APIs. Note that a class’ public methods does not constitute an API. Instead, publish an OSGi API bundle for a given functional object that is independent of the implementation logic. One of OSGi’s greatest strengths is its ability to compile against an API without having the implementation present.
The first step toward OSGi has been accomplished where an multiple projects can exist within a single application (within Buckminster/Eclipse). This separates iDempiere’s massive core from external process, callouts, etc… The next step is to go beyond Eclipse by using bnd and the full arsenal of OSGi technologies which includes the ability create an application from a collection of remote repositories. If iDempiere’s vision is to model itself after Linux, this would be a great step forward toward making real distributions (as opposed to repository branches) possible.
One of the best ways to see API-based OSGi programming in action is to complete of OSGi Enroute examples.
When Successful Other Benefits Follow
If we had a magic wand and were able to wish iDempiere into a OSGi bnd-enabled collection of API-based repositories, many of iDempiere’s current challenges and visions we be solved.
Testing
One of iDempiere’s greatest weaknesses is its lack of automated testing. This point is particularly painful for the iDempiere community because it does not follow a release process. Instead, they follow a branch process. iDempiere does not create releases; they open and close branches periodically. This means that building iDempiere off of what iDempiere calls 3.1 on any given day might produce different results depending on who committed what on that day. This effectively means that each day could be an upgrade. Daily upgrades without robust test suites are the foundation of technical horror movies.
The owner of a functional repository would be responsible for both API level and unit level testing. If you are the owner of ALL of iDempiere, testing is daunting because there thousands of classes and tens to hundreds of functional areas. If you are the functional owner of Order management, your life is much easier. You have a couple of classes and one functional area.
Versioning
One challenge of supporting an API is managing versions. What does it mean to produce a new version of the Tax API? What does it mean if others are still pointing at your old version of tax? OSGi bnd handles this situation quite well. When you release a new version through a repository, the old version continues to exist. This means that iDempiere instances (distributions) pointing to your old version will still continue to function.
Going back to the vision of modeling the iDempiere community after Linux and its distributions, OSGi’s version abilities brings a flavor of package management to iDempiere.
Looking Forward
What happens if nothing happens? …if iDempiere’s structure does not change, and everyone stays in line behind a single individual? Nothing really. iDempiere continues to exist as an obscure project that people happen to find out about. It won’t really grow or shrink. Community members will make enhancements. Some of these enhancements will connect iDempiere to other tools. No other tools will actively integrate with iDempiere. The business code that Jorg created in 2000 is good enough that it will probably never die; however, will most likely never reach mainstream either. History has proven both. It is not a bad life for a program to live.
ADempiere and iDempiere Strengths and Weaknesses
Carlos and Heng Sin are probably two of the best developers in the ADempiere/iDempiere community. They are consistent, write easy-to-read code, work efficiently and maintain a high level of productivity for an extended period of time. When these two left the ADempiere community, the AD community struggled to piece together a viable release. It took years. The ADempiere community was still producing features, they just could not package them into a formal release. Adaxa, the accounting and operations powerhouse, greatly contributed to ADempiere’s flurry of new business features.
The split of ADempiere and iDempiere was as if the business analysts went one way and the developers went another. The result was that ADempiere kept adding new business features and iDempiere re-created the application framework. This is not to say that Carlos and Heng Sin are not good analysts and the the ADempiere does not have good developers. What I am saying is that each group has clearly demonstrated a desire and a talent for one vs the other.
I cannot help but think what the world would be like if iDempiere’s framework strengthened to the point that business analysts from both communities could adopt iDempiere’s framework and drive business logic at a healthy technical distance. Let the developers drive the framework and let business analysts drive business logic. Sounds like a powerful co-existence. Sadly, such a scenario is not likely. Words have been said, and actions have been taken to make such a world an unlikely reality.
Up until this year, I have been extremely proud of iDempiere’s position in its community. It was focused on strength of skills and strength of people. The forum remained a focal point to innovation. An opportunity presented itself for iDempiere to help ADempiere this year. Rather than taking action to help, almost nothing was done. What was proposed would have caused much more harm than good. Unfortunately, both communities took a step back in this situation.
Open source ERP is an amazing concept. Let’s keep it moving forward.
What is the best way to Learn iDempiere and ADempiere?
I teach an on-line class that covers how to learn, configure and audit open source ERP. It uses iDempiere as the reference ERP. Here are the course frequently asked questions. I have learned much over the last ten years, and I have much to share. I look forward to seeing you there!!
Why consider Open Source ERP?
Open source ERP gives you every opportunity to prove or disprove its ability to support your company’s ERP needs on a timeline that satisfies your organizational needs. With open source ERP, you do not face the same financial constraints nor do you face the same conflicts of interest as with commercial ERP. Instead, you invest in the appropriate skills and knowledge for your people and processes. Best of all – if open source ERP cannot solve your company’s needs, you can safely justify spending the additional $2K to $5K per person per year for life of your commercial ERP to help drive your organization’s success.
Open Source ERP Round Rug Effect
Open Source ERP has what I call a “Round Rug Effect”. If you were to liken the ERP evaluation process to a 10′ x 10′ room, the story would go something like this:
Oracle, SAP, and Microsoft are a 10′ x 10′ ERP rug in a ten by ten foot room. They cover the room nicely. You will be hard pressed to find a feature or a use case that they do not cover.
Open Source ERP is like a 10′ round rug in a ten by ten foot room. It will cover the vast majority of the room; however, it will leave the corners bare. The questions are: “Do you live and operate in the corners?” or “Is open source ERP good enough?”. For most, the answers are “sometimes” and “yes”.
If you are in the ERP evaluation mode, you should ask yourself “Should I include open source ERP in my evaluation process?” If you are less than $300M USD revenue, your answer should probably be yes! This answer comes from these concepts:
Pillars of Cost – Since open source ERP is free, that means that all the cost of proprietary ERP should be allocated to the corners. If you use height to illustrate this allocated cost, the corners turn into tall pillers of cost.
Cost of Innovation – At first look, the price tag of free open source ERP is the most appealing benefit; however, this benefit soon becomes overshadowed by the flexibility of open source ERP. If organizational leaders take just some of the cost that would otherwise be spent on Oracle or SAP, and they invest it back into the organization’s skills and knowledge of how ERP works, operational efficiency will never look the same again. If you know how to change the system for the better, and you know it will work. Why would you not?
Monday to Monday Cycle – Business leaders drive innovation in a company. This innovation is no more apparent than in the traditional Monday morning business meeting where a CEO comes in and paints a picture of the next greatest thing. His or her next comments are “Will it work?” and “Make it happen!”. Open source ERP helps your business and IT teams say yes more often. You are no longer completely dependent on a high-priced Oracle Integrators. You are no longer dependent on spending 18% every year to Oracle for software that you have little control over. Your team applies its knowledge of the system and the knowledge of its world-wide resources to create a proof of concept that paints the real picture the following Monday.
Right Pay Grade – Open source ERP puts the right tools in the right person’s hands at the right pay-grade. there is little more wasteful that paying a $150/hr integrator for something a Jr IT professional should be doing. Open Source ERP removes the artificial barriers that exist in proprietary ERP.
ERP for Everyone – User licenses/seats are no longer a consideration. This point cannot be stated strongly enough. At first look, you might think this point is about saving money. It is much more than that. You now have the freedom and flexibility of allowing everyone in your company to interact the system that drives your operations. You simply assign the right roles to the right people to give them access to the appropriate information.
ADempiere vs iDempiere vs Openbravo vs Compiere
The ADempiere, iDempiere, Openbravo and Compiere environments are amazingly similar. iDempiere came from ADempiere. ADempiere and Openbravo came from Compiere. Compiere came from Jorg Janke. Jorg came from Oracle. As a result, iDempiere and ADempiere have much in common with Oracle’s ERP in terms of the financial feature set.
This is both good and bad. Good because iDempiere and ADempiere are quite capable to help a company grow beyond $500M USD. Bad because they tend to be more complex in that they account for multiple languages, accounting schemas, currencies, calendars, costing types, costing methods, etc…. If you are a growing organization, and you need a system that will grow with you, and you have the right internal talent/resources, iDempiere or ADempiere will be a big asset for you.
The biggest difference between these products is that ADempiere and iDempiere are pure open source. ADempiere and iDempiere make all feature available for free. Compiere and Openbravo hold back features behind a commercial or paid license.
Here is an article that discusses the differences between iDempiere and ADempiere.
iDempiere and ADempiere vs OpenERP
iDempiere/ADempiere (iD/AD) and OpenERP approach ERP from two very different directions. OpenERP comes out of the box with very simple options. If you are coming from QuickBooks, and you need a simple ERP system help you manage your business, OpenERP will look and feel comfortable.
iD/AD comes out of the box with every feature installed and configured to run a $200M+ USD business. If your business is growing rapidly, and you are willing to invest the time to learn an enterprise accounting system, then iD/AD will give you confidence.
Which one is best for you depends on your internal talent, growth and business complexity. Here is a post to help you learn more.
The post appeared first on iDempiere Training Open Source ERP Chuck Boecking.