[This post is a reply to Peter’s post here: http://forum.basecamphq.com/forums/8/topics/303]

Basecamp 3rd-party API developers should probably standardize on the format/syntax in which we add data to Basecamp. I know we’ve chatted a little about this before on the old forums, but I can’t remember what we had come up with. The [[name value]] syntax is good, though perhaps [[name=value]] might be better (avoiding using space as a delimiter). I’d want to stay away from colons because of confusion with time-information. Chris, what syntax are you using to tag items with QuickBooks catgories?

Peter, I agree.

I’m not tagging QuickBooks Categories in Basecamp, I’m actually doing a direct match between the value of the To Do Item (todo-item/content) and the Name of the Item in QuickBooks.

But, I am using bracketed metadata in two places:

1. In ChronoPipe, appending [QB] to the end of a time entry to show that it has been imported. On subsequent runs, ChronoPipe will ignore any entry with [QB] appended.

2. In BCTix, prepeding a ticket number such as [1234] to a Message.

In both cases, I am using single brackets because I think they look the cleanest of all options. I can understand why dbroekman may have used the double-square brackets—I sometimes find myself using the single square brackets for other things just in the course of using Basecamp.

I am planning a rewrite of BCTix, and will be doing some things like linking To Do Lists to Messages. I am debating whether to do this with a bracket method, or just track the link in a separate DB.

Another thing I’ve been thinking about is for a rudimentary resource planner, either using brackets to link two milestones (start and end) or store a start date in a Milestone, ie:

4/15/07: Paint deck [Start: 4/1/07 Effort: 8h]

Discuss!

Being a novice to all of this, my opinion doesn’t have as much weight, but personally I tend to like the single bracket approach. This is particularly more useful to me because the wiki system that I currently use for documentation uses links with the double bracket paradigm. Plus, don’t forget that Milestones have a 100 character limitation, so keeping less in the title is better than more.

/2¢

I’d avoid anything that’s even slightly likely to be misinterpreted (i.e. that a human might type in), which for me would include single square brackets. I already use double square brackets in other situations, so I wouldn’t have a problem with that. If efficiency is an issue, almost nobody uses curly braces {} or pipes ||, so that might be a better choice.

Stickman, good points. I will admit to using pipes || for some internal parsing to create custom arrays and sort orders, so that would be bad for me (but really I should change my internal functions to use non-printable characters for splits…)

Anyway, here is some more food for though:

I had a theory the other day that some meta-data might be able to be hidden in HTML <!- comments ->. I tested this morning and got the following results:

• To Do Items
• Milestones

• To Do Lists
• Time Entries
• Message Titles (oddly, this will hide the first part of the comment tag: <!-, but not the end tag: -> )
• Message Bodies

So in at least two areas the Meta data can be hidden from the Basecamp user in view mode (they’ll still see it on edit).

EDIT Beast is making strickethrus out of my comment tags, but I think you get the idea.

Interesting discussion!
I am working on a custom report system from Basecamp for our company, and I planned to use the [key 123] notation. It is more readable than double brackets or curly braces for a user that is looking at it in Basecamp, and easier to write manually for them if they want to do it from Basecamp.

I don’t think there will be a problem with interference from random notes with brackets the users might have added, it would have to match the keyword completely, and then it is probably just what the user intended :)

Interference with other plugins/addons to Basecamp might be more hazardous. We could use a namespace, like [qb:status=imported], but that would be tricky to write manually for the users, and actually looks a bit scary for a non-programmer.

It could in some cases be interesting to integrate with other plugins/addons, the [[est 3.5]] from the Avalanche widget might be useful for a reporting tool for example.

The biggest reason we used the double bracket was so there would be no problems parsing it out. It is very unlikely that a time entry would include anything in double brackets, but we’ve seen time entries with single brackets before. To be honest, we hadn’t really thought about interference with other add-ons.

I wouldn’t have any problems switching the format, but the only concern I have is that it must still be human-readable, so nothing like [[Ava:est 1.5]].

I think the easiest thing is just to agree on a tag’s meaning in a specific context, so for example agree that [[est .5]] in a time entry will be used to store time estimates. I don’t know if there is an easy way to keep track of all that, though (like for new developers so they can follow the “rules”).

Good thought with the comments, Chris. Shame it’s not consistent.

What would be really sweet is if 37s would build-in a <metadata> namespace tag to all items, which would accept arbitrary non-basecamp fields. It could simply be serialised for storage and returned with all requests.

OK, I don’t believe this will happen but it’s a nice thought…:)

I’m still wondering if curly braces would be better than square brackets. I’ve never seen anyone use them outside of code, so it should be pretty safe to use without the necessity of doubling them up.

As for keeping track of third-party tags and their meanings: well we could start a new thread and ask for it to be made ‘sticky’ (assuming that’s possible in this forum software). It would require someone to volunteer to maintain it (i.e. copy all new additions to the first post in the thread). Another blue-sky idea: 37s could create a wiki for API development. I already have a bunch of stuff I could post there (e.g. inaccuracies in the documentation).

Another blue-sky idea: 37s could create a wiki for API development.

Do I hear the beginnings of a “developer community” forming…?

I’d be more than happy to setup and host a developer’s community site with a Wiki and even additional forums… Caveat: it would be hosted on a Windows box running the DotNetNuke CMS framework (which is what we use in my main consulting company for most things) with Wiki and possibly Forum modules added. (Hopefully that platform doesn’t cause too many gasps amongst the crowd, it really is a good framework for something like this).

I have some VBScripts and Excel integration I’ve been meaning to post up for some time now and this would be a good home for it. Libraries by code example, forums by code type, etc. are possibilities for something like this.

Let me know your thoughts… I could have it up by the end of the weekend, but if someone else wants to take it up as a project, just let me know and go for it!

It’s a very kind offer Chris and I’d be happy to go along with it. I’d offer to do it myself but right now I have enough half-finished projects without adding another! :)

If we ask nicely, I wonder if 37S might even add a link to the site from the API page…I would think that it would be good for business to have an active developer community around the (eventually) various 37s products.

I’m not fully into the development within the BC API, but I think it’s a great idea & could serve as an excellent resource.

Sounds like a great idea to me.

I’ve been a code minkey for about 25 years, tho I have done nominal BC coding using PHP (I want to learn ruby). Would be great to learn from some of the senior BC/Ruby programmers that frequent herein.

=)

Aside – Chris: Yes, it would be really valuable to have a developer/community wiki. It would probably be good for it to be managed/maintained by 37s or their most active 3rd party developers. I would have no problems with you hosting it. But that’s a topic for a different thread. :-)

Back to the subject at hand: Wow. I had no idea the post would take off like this. Looks like the time is right.

It could in some cases be interesting to integrate with other plugins/addons, the [[est 3.5]] from the Avalanche widget might be useful for a reporting tool for example.

Exactly. If we standardize what we’re doing it’ll mean an exponential increase in value for all BC users. Plus it’ll increase the value of each of our tools—small sharp tools that work together.

I wouldn’t have any problems switching the format, but the only concern I have is that it must still be human-readable, so nothing like [[Ava:est 1.5]].

Natural vs Unnatural

• I think single “[” is more natural than double “[[”
• I think “(” is more natural than “[” which is more natural than “{”
• I think ’: ’ is more natural than ’:’ or ’=’ or even ’ ‘
• I think postfix is more natural for metadata than prefix

These are just opinions and I’m more than willing to bend on them in order to come to a consensus. I guess, in my opinion, the “most natural” syntax is something like “(est: 1.5)”. However, we might not want to go with the most natural (I could be very easily swayed to “[est: 1.5]” in order to prevent collision), though I think we should heavily lean towards the natural side of things.

Preventing Accidental Collision

The easiest way to prevent collision is to choose clear keywords (minimum 3 characters) and to require a particular position in the string. I would say all metadata should be postfix—though I’m open to allowing both prefix and postfix. If we do both of these things, I don’t think we need to be too concerned about accidental collision. As Rune Sandnes mentioned:

it would have to match the keyword completely, and then it is probably just what the user intended :)

For multiple pieces of metadata, I would say to space-delimit them like so: “(est: 1.5) (category: billable)”.

What do you guys think?

All reasonable points. I think single square brackets would be a good compromise. I suppose that, even if you do get a collision, the worst that’s going to happen is that a third-party application might interpret it in some unintended way, it won’t cause a problem within Basecamp itself.

Not sure why it would be necessary to space-delimit multiple metadata items, though. Surely the brackets are sufficient delimiters, so the space is redundant?

I think “(” is more natural than “[” which is more natural than “{”

I just pulled our company’s time report for April, and there are quite a few instances of using using (parenthesis) naturally.

I’ll admit that my opinion goes strongly toward single [square brackets]

I think that square brackets are natural enough to be typed if needed, but structured in their appearance so as to convey that there is something special about the data contained therein. To me, the parens are just to casual.

A “natural” syntax for metadata adds the huge advantage of being able to understand and even write metadata in the BC interface itself.

Agreed—I could see telling an employee “We use this app called ChronoPipe to bill the time from Basecamp, but if you do something that shouldn’t be billed, just tack “[QB]” to the end of your time entry and it will be ignored because ChronoPipe will think it had already been imported.”

I would say all metadata should be postfix—though I’m open to allowing both prefix and postfix.

I think in general postfix is best, but there are instances where I think prefix is appropriate such as putting a Ticket Number into a Message Subject (from BCTix):

“[1003] Backup job failing on Thursdays”

For multiple pieces of metadata, I would say to space-delimit them like so: “(est: 1.5) (category: billable)” (Peter Rust)

Not sure why it would be necessary to space-delimit multiple metadata items, though. Surely the brackets are sufficient delimiters, so the space is redundant? (Stickman)

A third idea: what about adopting a model simlar to CSS Styles:

[est: 1.5; category: billable]

I would much rather have to detect only one block of meta-data, then “know” how to parse all the data inside it. This could also eliminate the prefix/postfix question if we just said that the generally accepted way to parse meta data is to detect a [block] anywhere within a string and have your app attempt to parse what’s inside as it sees fit. Who knows, someone might come up with something crazy that puts a bunch of blocks in a Message body.

With a wiki, we could have a standard definition list of what keywords are chosen and people can just add what they’re using and document how their app uses it.

I think it would be easier for a developer to only relate to “his/her” metadata, if there is a [code] i need to change, I shouldn’t need to put back the rest of the contents of the bracket. So, my vote goes to square brackets (because they are “special” and still easy to type) and separate blocks for each metadata item.

Example [NB] and [est: 1.5]

(space between is optional, placement in the string can be anywhere, post-, pre- or middlefix)

I am currently using [NB] for Not Billable, because it is very quick and easy to write by hand. Unfortunately it is also read as Nota Bene by many.

We currently only need “Not Billable”, not “Billable”. (Non-deliberate attempt at confusing sentence)

Billable/non-billable status seems to be the most important to standardize in my opinion.

My only concern with single brackets is that Basecamp already uses them in time entries (though not really for metadata). If you have a todo item associated with a time entry, and you then delete that todo, it adds “[for deleted todo: blah]” at the end of the time entry (I didn’t look up the exact syntax, but it’s something like that).

Then again, as long as you have the keyword to search for, I don’t think it’s that big of a problem.

I disagree with the idea to use one block of metadata, like Chris suggested. I know that I’m using regex to parse out the time estimate, so this would increase the complexity significantly (since you can’t just search for /\[\[est/ anymore).

I like the idea of the developer wiki, though. However, if 37S really cares about us developers (and they should if they want their platform to take off), they should host it on their site.

Chris Busse: are you still happy to set up the wiki? No hassle, but if you’ve changed your mind for whatever reason then please let us know so alternative arrangements can be made. I can probably set up a wiki on one of my domains without too much effort.

I am—giving it more thought I realize there were about three wikis I needed to setup for various purposes. I went ahead and got a Stikipad PowerUser account which supports unlimited wikis, domain mapping, and Textile and seemed friendlier than the .Net solution I was originally looking at.

I’ll be posting up an announcement in this forum under a new thread as soon as the DNS change goes thru!

Excellent. Thanks! Looking forward to getting this thing going.

Chris: Thanks for setting up the wiki! That’ll be real valuable.

• Use single square brackets
• Use a colon to separate the key from the value (key should be at least 2 characters)
• Put multiple key-value pairs in their own square brackets
• Delimiters don’t matter between multiple key-value pairs because they can occur anywhere in the string

• What about a space after the colon? (is “[est: 1.5]” ok? or does it need to be “[est:1.5]”?)
• What about single keys with no value? (“[QB]” or “[NB]”. Should we allow this? Is an :true implied: [NB: true]. This is certainly a fast and convenient short-hand for frequently-used items, but it seems we might want to restrict this to the most common use cases. I wouldn’t want the “namespace” to be too cluttered with various “tags” that could be [tag: x] or [category: x]. Maybe [QB: imported] is clearer than just [QB]?
• What about case? (I think metadata should be case-insensitive)

Peter, on your three questions my preferences would be:

1. Space after colon is okay—its readable and easy enough to LTrim in your language of choice
2. I am not quite impartial on this one, but I would like to allow single keys with no value – checking for present/not present | true/false (true implied)
3. Case-insensitive for sure.

I’m happy with both Peter’s summary, and Chris’s answers.

Formalised here

I’m happy with Chris’s answers too.

dbroekman, Rune Sandnes: are you guys ok with the syntax?

In the absence of objections, I’ve added a couple of pages with example code (C# and PHP) for creating and extracting meta data tokens. Jusat a quick first pass, any feedback appreciated.

The syntax is perfect for me. And good work on the Wiki!