Repository navigation
Switching to the open-core model #1170
Description
Dear markdown hackers,
Thanks for your long-standing support.
Reflecting on what we learned over the past one and a half years, we decide that we want to move CodiMD and HackMD into the open-core model to better address the needs of the community.
What did we learn?
Back in October 2017, when we forked HackMD enterprise edition (EE) from HackMD community edition (CE), we had two goals: In addition to supporting ourselves, we wanted to keep the HackMD open source community active.
To achieve 10BC8 the second goal, we promised not blocking any pull requests, even those with features similar to HackMD(EE). We kept patching vulnerabilities and fixing bugs. And we invited two community members to be the community's advocates. They did a great job moderating CE and making sure we kept our promises.
Still, due to the lack of clear communication with the community at large on our part, the confusion between the two versions persisted, even after we changed the name of HackMD(CE) to CodiMD. Issues were posted in the wrong channel, same questions regarding the forking decision got asked over and over again, and extra burden were placed on our community advocates to explain the naming, the difference, and the history between HackMD and CodiMD. Moreover, the uncertainty of our continuous support for CodiMD deterred contributions. We've seen the activity of the community decreased.
From studying other successful open source projects, we learned that for an open source community to thrive, continuous support and maintenance promise from an (often commercial) entity is essential. It is imperative that we show strong commitment to the project, take full responsibility directing the open source project as well as the commercial one, and keep the fine balance between the two goals.
What do we mean by open-core model?
The HackMD production service at hackmd.io and HackMD(EE) had gone through major refactoring since the fork. It is now written in Typescript for easier maintenance and its performance improved to support the users on our production server. We will create a new "HackMD-core" repository, which contains the core code base of HackMD. We will then refactor CodiMD and replace its core with the HackMD-core.
Current and future features on both HackMD and CodiMD will be offered as modules that could be plugged into the core. All current CodiMD exclusive features will be refactored as modules and offered with the core as a work-out-of-box bundle. These include:
- Dark mode
- CLI
- Minio image upload
- OpenID authentication
- MatterMost Authentication
Some current HackMD exclusive features will be added to the CodiMD bundle as well. These plug-ins will also be open source. They include:
- Book mode
- Profile page
- User settings
- Embedded note
- Speakerdeck rendering
- Google drive integration
We will uphold the following principles during and after the migration of the CodiMD project:
- the database should be backward compatible so there would be no worries over existing CodiMD content.
- the code base should be modularized and API would be clear so the community could build cool things on it.
Open-Core Road map
Here's our preliminary road map to the migration:
- Modularization pilot:
- Authentication module
- Book mode
- Core refactoring and performance improvement
- Modularizing:
- Profile page
- Full-text searching on profile page
- Settings
- slide mode
- Publish mode
- Editor mode features
- Embedded note
- Permission improvement
- Change notification
License
The HackMD-core repository and all free plug-in modules will be licensed under Apache 2.0, while paid modules will be licensed under our own commercial license. The license of CodiMD and CodiMD exclusive features will have to remain AGPL v3 even after we finish refactoring.
Discussion
We want to thank the community for your patience over the past one and a half years. We hope to better support the community by this switch to the open-core model. Please give us a thumb-up emoji if you think this is a good idea, or let us know how you think by leaving a comment below.
Activity
davidmehren commentedon Mar 16, 2019
I like this. Even though it's yet another change to the relationship of CodiMD and HackMD, a "standard" open-core model is way easier to explain to people.
Regarding the repo-structure: I understand there is going to be one HackMD-core repo. Is the current CodiMD repo going to be obsoleted? Will there be one repo for each plugin or one big monorepo for everything?
ccoenen commentedon Mar 16, 2019
There's power in the choice of AGPL-v3. And while I understand the reasoning of choosing apache license from a business point of view, this is the point that's keeping me from jumping in joy right now.
I would also like to enquire how "the community activity has decreased" is measured?
kindfulkirby commentedon Mar 17, 2019
Won't CodiMD remain under AGPL, though? I'm not sure there is a downside to opening HackMD to an open-core model, but I might very well be missing something.
(And I'm just a user, not a dev 😉 )
ccoenen commentedon Mar 18, 2019
The currently available CodiMD codebase remains under AGPL, that's right. There's two things that worry me about this. First:
This would mean that this will be the end of CodiMD (AGPL) based on this repo under this organisation here. Of course, anyone is allowed to fork and use the code that's here right now. But it also sounds as if the HackMD team also intends to run the open core under the name "CodiMD".
This is a somewhat unprecedented step, and I believe this must be an error introduced in translation or something like that. All open core projects I know run under something like "MySQL Community Edition" or "Zimbra Collaboration Open Source" or "GitLab Community Edition". I believe
replaceto be a typo for two reasons: It would basically squash the existing development effort under AGPL but also dilute the HackMD brand. Basically everybody loses, and I don't think that's intentional.So we come to the second thing that worries me:
Let's assume we have HackMD-CE (Apache) and CodiMD (AGPL) from now on going forward. The licenses are incompatible[1] and prohibit any easy exchange of code. Basically, all work that's done on one can't (without extra work by the individual author) be applied to the other. While I would object to "decline in community", it is still a small community. And this community would surely split between those who keep using CodiMD (AGPL) and those who want HackMD-CE (Apache). So, in essence: CodiMD (AGPL) may well be at stake.
[1] Aside: the licenses:
This is not a fault of either license, nor is it an accident. It is part of their different design goals. As stated before, AGPL gives a lot of power to the community and to its users. One of these powers is, that the code of your service can't be hidden from you. You as a user (any user! Not developer, not Admin. Any regular user!) is entitled to a copy of the source code. Apache is very different in that regard.
SISheogorath commentedon Mar 18, 2019
Hello Community, hello HackMD team,
First of all I want to tackle the problem from a very practical perspective: AGPL is complicated, AGPL pushes free software, AGPL is great. When we have an Apache licensed core, and the CodiMD parts remain under AGPL, then the end result is still licensed under AGPL.
That's how it works and that's why we originally decided to use AGPL. This means that whenever you decide to keep any plugin under proprietary license, it becomes incompatible with all CodiMD plugins. In other words people either run a CodiMD-only version or HackMD re-implements all features in CodiMD. Effectively, it either destroys the purpose of this issue's idea or it makes CodiMD obsolete in general which makes the whole idea of opening an issue here, except for marketing purposes, pointless. Sorry for stating it that clear, but boils down to that, and I already pointed that out in the previous conversation we had, even when I wasn't that clear on this.
One of the reasons why I'm personally far from happy with this idea in general, besides the license issue, is that it literally throws away more than a year of work by the community in order to bring HackMD and CodiMD together with the intention to monetize the CodiMD community. Which is in a way pretty much against what you purposed in 2017 when you decided to let the community organize itself.
To be clear, no one wants to prevent you from making a living, but it should be done in a fair and reasonable way and this idea is basically "Let's change the license from AGPL to Apache" (of course by taking care of the problem, that AGPL in general doesn't let you do that, which is tackled by bringing a core component under a different license).
And you as HackMD team, as well as I, know that this idea didn't came from nowhere. It came up all of the sudden, when I asked you about your feelings about moving this repository along with other repositories to an own CodiMD organization, since the community is organizing itself anyway.
I would love to see more contributions from your side, but I understand that you have to focus on making a living with HackMD. What I don't understand is what hinders you on pushing your changes to CodiMD instead of an own core repository. Sure, things would be AGPL licensed, but since you are the copyright holder of your parts, it wouldn't matter when you run that parts as a proprietary product on HackMD.io. The only difference would be that you can't take the advantages of community contribution, but that again would allign with your commitment from 2017 about taking the communities fruits.
davidmehren commentedon Mar 18, 2019
After my initial positive comment I thought about the other comments and AGPL in particular. I must say, I probably misunderstood the goal of this issue.
I have to concur with @SISheogorath and @ccoenen. You can't just take over a community project and throw all work that has been done out of the window.
I somewhat understand the "with AGPL we can't make money with proprietary addons" thing. But OwnCloud tried that and (at least I would call that) failed. Nextcloud explicitly does not use an open-core model but solely makes money by selling support around Nextcloud. Maybe something like that would work for you?
kindfulkirby commentedon Mar 18, 2019
Ah dang, you're right, I missed the part about how they'd change everything but "CodiMD exclusive features" to be Apache licensed… I figured it would be dual-licensed and remain AGPL (or triple, I guess, if they want to sell some of it?)!
jackycute commentedon Mar 18, 2019
OK, let me clarify this.
What we're proposing is to keep the CodiMD and all previous contributions' license "as-is" (AGPL).
And we will extract the HackMD-core as a separate repo (licensed under Apache 2.0) from HackMD (EE) which will be refactored and open source.
Then we will try to refactor CodiMD to use HackMD-core and make all features pluggable (all CodiMD's plugins will be licensed "as-is", AGPL).
I believe the licenses will be compatible in this way.
Also, we will keep maintaining HackMD-core, so that CodiMD can use up-to-date, enterprise level core and install or write any plugin that they love.
From above points, you can see that HackMD (EE) is the version that installed exclusive paid plugins (close source) that extends HackMD-core.
And we hoping to open source those paid plugins one day and this proposal can make that possible when the time comes.
SISheogorath commentedon Mar 18, 2019
@jackycute well, while it may is not intended, but at the end of the day it would mean that we would swap out large parts to the current code base in order to make it compatible with hackmd-core. And while our codebase is AGPL licensed the hackmd-core would be Apache licensed.
In other words, large parts of our codebase would change their license from A 8 GPL to A 10BC8 pache. Maybe not with exact intend to do this, but at the end of the day, that's the result.
And yes, we could integrate an apache licensed core into CodiMD but the question is: What is left from CodiMD when we swap out the core? Looking at the features that were listed above that would be 5 JS files, let's be grateful and they we end up with 20 things that are CodiMD exclusive. This would basically destroy the foundation of the community around CodiMD, which has a strong relation towards free software, in order to move people towards HackMD-core which runs on a simple permissive license.
And to state it clear, nothing is preventing you from open sourcing features like book mode or user profiles. It just doesn't make a lot of sense, from a business perspective, which is probably the reason why it didn't happen, but it's also uncool to bring it up here like you couldn't do it otherwise.
pferreir commentedon Mar 18, 2019
I think there is a choice here, between:
This is my personal opinion: as already discussed on Matrix, as a user and occasional contributor, I think there is already enough evidence that while Open Core does tend to work finance-wise (so, from the PoV of sustainability), it also alienates a great part of the FLOSS community and increases the likelihood that larger players will just try to crush you. Look at ElasticSearch and what is now going on with AWS. Or MongoDB.
But those who contribute the most should be the ones taking the decision (@SISheogorath in particular).
SISheogorath commentedon Mar 19, 2019
Okay, maybe all this got a bit dramatized yesterday, sorry for that, but I wanted to make a hopefully reasonable point. Let's come down to a bare technical question:
@jackycute what do you expect that CodiMD repository will look like after the refactor?
jackycute commentedon Mar 20, 2019
Thanks for all your feedback, they are well received. The fundamental reason we started the discussion of the open-core model was hinted in the issue above, but probably not as clear as it should be, or got overlooked. Let me try another take here: We think CodiMD deserves more love from the HackMD team. Now the question is how?
These are our constraints:
Therefore, open-core model seemed like a great solution to us.
Yet, your sincere feedback forced us to come down to the very essence of our assumption. Is open-core model really the only way? AGPL does provide very good protection and assurance for the community, can we love CodiMD more while respecting that? Lastly, nobody likes the project they had worked on being hollowed-out (even it's replaced with a better version), we have to respect that as well.
So we think of another way:
Hopefully this addresses most of your concerns that were valid. Let us know if you like this plan better by giving us a thumb-up emoji, or leave your comments below.
pferreir commentedon Mar 20, 2019
@jackycute it's great that you have listened to the community's concerns and that you're considering taking the harder path to keep us all happy. Thanks!
At the same time, there's something that has never been very clear and is even less clear to me now: who owns CodiMD? Is it the Open Source sibling of HackMD? Is it a community project that HackMD happens to contribute to?
That may seem like a detail, but it makes all the difference with regards to governance.
13 remaining items
pferreir commentedon Mar 25, 2019
@GitOnion I'm not sure what you are trying to imply here, but I was pretty clear in saying that my participation has been modest and that others would be more qualified to speak. I only asked for clarification on something which I find important, which is the ownership of the project.
But, BTW, my participation still amounted to 2 PRs (one of them quite important: generic OAuth support). I did try to attend the community call on one occasion, but it was postponed for lack of participation. I'm not sure if that fulfils you community participation criteria, but I do think I have the right to say one thing or two 😉.
GitOnion commentedon Mar 26, 2019
@SISheogorath,
It doesn't matter how you phrase it, taking over something other people own is never a good behavior.
Three things:
If people aren't even willing to "bother" re-subscribing to a newly forked repository (as simple as just clicking some buttons), then our current governance probably is NOT bothersome enough that you have to move to an own organization.
How do you know (and would prove) that "all" community members are willing to move to another organization? There are close to 150 watches, 4,600 stars, and 600 forks. Does EVERYONE, or even half of them, knowing HackMD team is where it started and has better supports, want to move with you? Also note that these numbers are the "real" user community we serve, not just people leaving echo chamber comments above.
Now that I've made it clear that the community already has what it asks for (clear ownership), if you still decide to fork the code base and tell the community to go "exodus" with you, it's still your freedom that I cannot and would not limit.
It's interesting that you argue that running a proprietary product based on the same original source code in order to make our living dis-qualifies us to be the governors. This is a common practice: Docker, Elastic, GitLabs, to name a few, runs the same model. There will always be fundamentalists complaining about their governance, but in my opinion, these projects have pretty good communities.
@ccoenen,
This is a software project, code is certainly a major part of it. At the end of the day, a software project is consisted of lines of code. I will one last time point to the fact that the majority of code is contributed by the HackMD team. If you think the work @SISheogorath done over the past and a half years is significant, then when you look at how his code contribution compares to that of the HackMD team, how can you not see how valuable is the HackMD team?
I really don't see why moving the project out from our organization would be a great move for the community. Indeed, the HackMD team was short in time when #531 came out, and we really really are grateful for @SISheogorath's contribution to make the community thrive, but this never have to be an either-or problem for any of us in this community to choose from. If the project stays, the community gets BOTH.
@pferreir
My point was only to show that the HackMD team is every bit as much, if not more, a community member as you are. Which, in you previous comments, you seemed to believe otherwise. If HackMD team is part of the community, then this project is already community-governed, as many of you wanted, that's the whole point I wanted to make.
And for your PRs, we are very grateful as well. Everyone in this community is encouraged to say things they feel important, and the community will try to accommodate those if it feels those are important as well. That is the spirit we had when we open source HackMD, the spirit we have today as we discuss our perspective here openly, and the spirit the community can be assured that we will continue to hold in the future.
ccoenen commentedon Mar 26, 2019
@GitOnion I'll give this a couple of days to think about this.
jackycute commentedon Mar 27, 2019
Hi guys, thanks for all your responses and discussion.
I think those might be gone too far of what we’re proposing and also off the intention.
There is no point to go further with this thread, let’s stop right here.
Thanks @GitOnion for such detailed and great explanation.
Thanks @pferreir for bringing up questions about the governance.
Thanks @SISheogorath and @ccoenen for their hard work and dedication for the community, I’ve been very grateful to yours contributions on both codebase and making a live community.
First, our intention was just trying to find a proper solution to maintain contribution from HackMD to CodiMD.
That isn’t easy since we don’t have the resources to maintain two codebases.
We thought there is one possibility (and people told us so) is open core model. But we didn’t know it might still have other issues.
Nevertheless, the comments on the Riot and here told us their concerns and we learn those now.
I think I understand the need to keep all the software freedoms (refer to https://sfosc.org), and license under AGPL will help this go on.
Second, the reason why we open source HackMD is to share the technology we made and let people to spread their creativity with this.
And we can devoted our self on the enterprise needs then up sale those users.
But that wasn’t the intention why @SISheogorath and @ccoenen wanted to help us to maintain the community and I misunderstood that back then.
Thus, I believe the current community consists of at least two different users and here is my desicion.
For the people who want to use the latest feature and knowing there is a company behind this project, you can stay here.
For the people who want to keep the code all away from business, I think @SISheogorath and @ccoenen will fork a new home for you.
I felt sorry about not making this very clear before and I’ve learned a lot from the community (from you!).
Thank you all for reading this and thanks for your time and contribution, sincerely.
P.S. Please do leave a comment if you feel something unclear, I will answer you as much as I can.
Happy hacking!
ccoenen commentedon Mar 27, 2019
To be clear: I did (and will continue to) suggest HackMD.io to people that are looking for a hosted, professional offering. While I can't know if any of these suggestions resulted in revenue to you, I really wish the HackMD business all the best and a solid, steady, good flow of income.
jackycute commentedon Mar 27, 2019
Thank you @ccoenen
HerHde commentedon Mar 28, 2019
Wow, that escalated quickly. So is it seriously a fork at @codimd then, @SISheogorath?
ccoenen commentedon Mar 28, 2019
If you've been using CodiMD so far, this move will probably not make much of a difference to you. Stars start at zero. Issue tracker is empty. Other than that, it's the same project.
I think it's a technicality if you think of it as another fork or not, but from a practical point of view there's still an Enterprise Project (HackMD EE) and a community project (CodiMD). Those were seperated over a year ago and both did fine in my opinion. Both will most likely continue to do fine.
HerHde commentedon Mar 28, 2019
That's it. But what will happen to this repo then? Will it be renamed to HackMD CE? Will the HackMD team maintain none, this or contribute to both? With the current core?
It's always somewhat sad to see projects splitting up, but I guess for many parents it's also hard to see the children leaving the nest, to bring up a metaphor here.
jackycute commentedon Mar 28, 2019
This repo will be maintained by HackMD team afterward.
We will say more about our future plan soon.
SISheogorath commentedon Mar 28, 2019
A vision for CodiMD
Here we are! CodiMD is running in an own organization. Hard forking is usually the last resort for an open source project, but we felt that it was neccessary for continued development and the future of CodiMD. If you wish to read up on the final debate that lead to this, please refer to the old bug tracker.
It's time to stand together and find our way through the jungle. In some regards, we are back to zero, and that's sad. But on the bright side, the issue tracker has a lot room for your problems! 🎉
Let's work together!
The new GitHub organization provides us with all the flexibility we need to work on our own terms. We already sent out invitations to various community project participants and will welcome people to join us.
You might wonder about the next steps. We currently rebuild the infrastructure parts that were in the hands of HackMD, like the container repository, and work on integrating services with our new home, which was previously more difficult.
With this done, we look towards a very interesting future:
One of the first actions we'll take is providing a fully documented and working API for CodiMD, which should allow you to import, export, create, delete and even modify your notes. You are welcome to participate in the debate about the API design.
APIs are great, but your eyes deserve some love, too: So we also started working on some UI/UX improvements. User Profiles will be continued and completed over here.
Finally, we also want to work on improving the connections and collaboration within the community. Our community forum should be up in a few days.
I'm looking forward to your ideas, inspiration and comments. Let everyone know, that CodiMD is not dead, it's reborn!
wolftune commentedon Apr 3, 2020
Interesting to encounter this history after-the-fact. I only just learned of CodiMD.
I wanted to clarify a misunderstanding I read above:
@pierreozoux wrote:
This statement shows a fundamental misunderstanding. AGPL is absolutely an Open Source license, period. AGPL is an approved license at https://opensource.org/licenses/alphabetical as are all the FSF/GNU licenses. Open Source does not mean non-copyleft. https://opensource.org/faq#copyleft
The issues with libre software are about giving freedom to users. Certain norms about project governance may be common, but all that matters is that users have full freedom, whether or not they ever do anything to help the project. That is the philosophical difference. Open Source focuses on how the project enables collaboration. Free/libre focuses on spreading freedom, not locking end users into restrictive terms. Both movements support nearly the identical list of licenses though.
zeigerpuppy commentedon Apr 8, 2020
Just weighing in a little here. I am a user and service provider who hosts a codimd server that is fully integrated with Gitlab SSO and Mattermost. I got interested in the project around the time that the community was making a decision to change the name of the project so that it was different from HackMD.
My understanding when this name changed occurred was that the project was taking an explicitly open source position, to differentiate it form the commercial project. That is really important for some users (including me). We certainly don't forsake companies making money from code, but once it's under AGPL, it's also a greater resource and can develop into tools and use cases that the original developer may not have imagined (or sanctioned!).
I don't think there was ever the expectation that the HackMD team would maintain the code going forward, we're already greatful enough for their decision to open source the code. It's also important for the HackMD team to realise that it's not possible (nor, would I argue, ethical), to then attempt to close the code by taking over the core with a different license.
There are great gains to be made by working together, but sometimes the motivations diverge. As I see it, one of the core aims of open-source software is to protect the future of the code from such changes in motivation, financially driven or otherwise.
I do hope that its possible to work together in the future and, as a humble user (and occasional bug tester) wish the communities the best for the future and humbly thank the programmers for their contributions.