aseem

zenkat: wikipedia mining

2008-08-30T02:34:41.0017Z

hi brian, i loved your talk on mining wikipedia. i'd like to mine the soft data, is there a smart way of parsing the unstructured stuff? like small, adhoc stuff -ie for pages in [[category: racecar drivers]] find "sponsored by _____" ?

you're cool.

i applied for an internship but didn't hear back. so i suppose i, and the idea on a limb.cheers-

tristan: Question

2008-08-30T00:03:01.0005Z

Dear Tristan,

I noted that you posted the bio of me (Anthony Glise) on Freebase. Thank you, by the way.

I have a problem in that one of my CDs (Partitas of G.A. Brescianello) has been placed on Freebase for free download. Since my split with Dorian Recordings, I secured all copyrights and performance rights to that and other CDs. I can't find a way to get directly to the Freebase staff to have that download pulled and the management company is getting VERY agressive insofar as persuing the matter - which I would obviously like to avoid.

As an active Freebase member, what would you advise so I can get this matter addressed before it gets out of hand?

Thank you,

Sincerely,

Anthony GLISE

robert: Is this image copyright free?

2008-08-29T22:17:30.0000Z

benjamin, here are two queries I threw together yesterday which might help you.

The first one gets a bunch of topics and any images associated with them, then tells you the license those images are under:

[
{
   "/common/topic/image" : [
     {
       "/common/licensed_object/license" : [],
       "id" : null,
       "name" : null
     }
   ],
   "id" : null,
   "limit" : 20,
   "name" : null,
   "type" : "/people/person"
}
]

This second one is roughly the same, but excludes licenses of a certain type -- in this case Fair Use:

[
{
   "/common/topic/image" : [
     {
       "/common/licensed_object/license" : [
         {
           "id" : null,
           "name" : null
         }
       ],
       "id" : null,
       "name" : null,
       "nofu:/common/licensed_object/license" : [
         {
           "id" : "/en/fair_use",
           "optional" : "forbidden"
         }
       ]
     }
   ],
   "id" : null,
   "limit" : 20,
   "name" : null,
   "type" : "/people/person"
}
]

robert: Is this image copyright free?

2008-08-29T20:30:04.0012Z

There are examples and documentation in the help center. If you only need to look at a couple of images, you can do it manually; the licensed pictures have a little license icon at the bottom. Unfortunately, some of the older images (like this one) were loaded before we required a license assertion for each one.

If you are loading many images at once, I would imagine that your application would require some other MQL usage anyway just to find them.

mediflick: Welcome, and congratulations!

2008-08-29T20:17:44.0012Z

Click the Help link at the top of this page and then look at the Screencasts in the lower right. I think those may be what you’re looking for.

mediflick: Welcome, and congratulations!

2008-08-29T19:16:45.0005Z

Hello, wow a message :) nice to see. Thank you for the info, it was hard to insert data, not knowing how the site works, but i managed it after a long time, maybe introduce a quick video for users to see what and how to edit the pages. :))

robert: Is this image copyright free?

2008-08-29T18:37:49.0012Z

Totally unfamiliar, but I do know other similar languages, so if you had an example that was close, I could disect and repurpose it. Thank you!

Business: Company type

2008-08-29T16:22:47.0012Z

Well, I guess it depends on what we want to use it for. Do we want to use generic terms for multiple legal types, or do we want to be more specific.

For example, PLC is Public Limited Company. "Public limited company" could be used to cover all those types in various countries, or it could be for the specific Plc used in UK.

But unless you are intimately aware of what SA (Societe Anonyme) is, or use the Wiki page to see what approximate equiv is, then chances are no one is going to use this field (at least for "harder" ones). I deal with company names all the time, so I'm familiar with Oyj, AG, KgaA, etc, but I don't know what they translate to.

Either way we're missing values. If we keep the "generic" route, then we (I) should comb through the wiki page and identify the big ones we're missing, like "Limited Partnership with Shares" (for SCA & KgaA, for example). Though I'm not sure if all can be mapped (like KK, which is common in Japan, though I think it is usually seen as Co Ltd).

So I think we need to figure out what direction we want to take, generic or country specific, and I don't think either is an easy answer (either you have a long list of specifics or a hard to determine list of generics).

mediflick: Welcome, and congratulations!

2008-08-29T15:47:06.0012Z

Sorry: your business. I wish we could edit previous posts… d-:

mediflick: Welcome, and congratulations!

2008-08-29T15:46:11.0017Z

Of several would-be business promoters to come to Freebase, you are among the first to actually use the site correctly. We welcome all factual, well-structured information about business enterprises, and encourage business owners and employees to fill out information about their businesses. Why not tell us where you business is located, who its founders are, etc.?

Welcome, and we hope you find some time to add information about other topics of interest to you, too.

Business: Company type

2008-08-29T15:20:31.0012Z

Enumerated types like Company Type aren’t locked, they’re just a little harder to add to, in order to discourage accidental proliferation. Additions must be made through the query editor or some other way that isn’t the client. We do encourage you to discuss the proposed additions with the community first, though, to avoid edit wars on high-profile types. What specific instances do you think are conspicuously missing?

www.VideoMenu.com: Video Websites

2008-08-29T14:28:06.0017Z

Please let me know about any new video websites or new web video technology. Thanks

Business: Company type

2008-08-29T14:00:35.0017Z

Why is Company Type locked? There are many legal structures of companies, and even state-owned enterprises. But very few are listed here. Is there a particular reason for this? If so, it would be good to have that documented so those of us filling in company information know how ot complete this field. Thanks

Olympic Games: athletes by country: Beware of Long Lists!

2008-08-29T06:36:47.0012Z

No worries. As a programmer, I probably should have realized the limitation of what I was trying to do and in the end it wasn't nearly as bad as I thought.

Help Center: to get (image) statistics

2008-08-29T06:31:11.0012Z

Hi parity, minor point: you don't need to send your authentication cookie for doing mqlread (only for mqlwrite).

Help Center: to get (image) statistics

2008-08-29T05:33:55.0012Z

I suspect /api/service/mqlread is returning a timeout or error. My Perl is rusty, but the until loop doesn't look like it's checking for a 200 OK return response; it just looks like if you don't find a cursor, you assume the counting is complete and exit.

Help Center: Batch update data

2008-08-29T04:42:45.0012Z

The only real "batch update" tool in Freebase is the list importer. If there were a link from the timezone to "places in this timezone" then you could do "import list" to do what you ask, but since there is no such link, I think you'll probably have to use the API.

Olympic Games: athletes by country: Beware of Long Lists!

2008-08-29T04:40:52.0012Z

Yeah, the import tool can be a bit of a pig on long lists. I usually try to do it in batches, myself. Sorry it caused you trouble :(

4. Metaweb Read Services: c# example

2008-08-29T04:26:34.0012Z

Thanks, weblivz! Probably the best place to put this is under Application Developer Tips, but I'm copying our docs guy, Phil, on this so he can help you out.

robert: Is this image copyright free?

2008-08-29T02:41:42.0012Z

Yup, that's certainly true. If you're looking for stuff that's absolutely OK for commercial use, I suggest you use a MQL query to look into the license on an image, and limit it to those which have licenses that fit for your purposes (i.e. PD, CC-BY, or whatever). I can help you with that if you're unfamiliar with how to do it.

robert: Is this image copyright free?

2008-08-28T22:53:19.0000Z

Thank you for looking into it! Fair use for me is a bit different than copyright free (or some CC flavor) because it would impact the commercial use possibilities.

Help Center: to get (image) statistics

2008-08-28T22:47:07.0005Z

The following is the code I used. Have a look only if you're free. I print out $count to see number of topics with images and $count~438

Afterwards I grep entire query result and got numbers close to you:

692001 topics with image

626805 topics with 1 image

62827 topics with 2 images

2210 topcis with 3 images

and so on...

#!/usr/bin/perl
use URI::Escape;
# This module provides the uri_escape function used below

# Build the Metaweb query, using string manipulation
# CAUTION: the use of string manipulation here makes this script vulnerable
# to MQL injection attacks when the command-line argument includes JSON.
#$band = $ARGV[0]; # This is the band or musician whose albums are to be listed

my $cursor = 'true';
my $query;
my $pre =
'{
    "qname":{
        "cursor":';

my $post = ',
        "query":[{
            "id":null,
            "limit":1000,
            "type":"/common/topic",
            "image":[{
                "return":"count",
                "id":null
            }]
        }]
    }
}';

my $allresult = '';
my $count = 0;
my $result = '';

until($cursor =~ /error/){
#while($cursor){
    $query = $pre . $cursor . $post;

    $escaped = uri_escape($query);

    $baseurl='http://www.freebase.com/api/service/mqlread'; # Base URL for queries
    $url = $baseurl . "?queries=" . $escaped;

    $auth = 'metaweb-user=###Enter Your cookie data here###';

    #$result = `curl -s --cookie \'$auth\' $url`;
    $result = `curl -s $url`;


    $result =~ /\/api\/status\/(.*)/;
    $status = $1;

    $result =~ /cursor\":\s*(\".*\")/;
    $cursor = $1;

    $count = $count + 1;

    print $result . "\n";
}

# Use regular expressions to extract the album list from the HTTP response
#$result =~ s/^.*"album"\s*:\s*\[\s*([^\]]*)\].*$/$1/s;
#$result =~ s/[ \t]*"[ \t,]*//g;
#print $allresult ."\n";

print 'done'."\n";
# Finally, display the list of albums
#print "$result\n";
#
# vim: ts=2 sw=2

robert: Is this image copyright free?

2008-08-28T21:26:54.0012Z

I suspect it was uploaded under the terms of fair use, but for some reason the image page doesn't say so. I'll check with the guy who uploaded it (I'm copying his user profile on this thread.)

Help Center: to get (image) statistics

2008-08-28T20:34:47.0012Z

Well, I got essentially the same results as reported above using metaweb.py. Are you using your own Perl module, or one from our API library?

robert: Is this image copyright free?

2008-08-28T16:28:34.0018Z

Looks similar to the corporate stock photo, so I wanted to double-check the copyright status. Thanks!

Help Center: to get (image) statistics

2008-08-28T15:19:55.0012Z

Thanks. I counted the number of iteration for the code above and multiply it by 1000 (since I set the limit 1000) to get approximate image count.

Help Center: to get (image) statistics

2008-08-28T05:53:50.0012Z

Some MQL reference material can be found here. I iterated through topics with images and then recorded the number of links there were. Currently, you can't constrain the query to ask it to return topics with exactly n amount of images. I'm not sure why your results differ so greatly that mine - I'll try to investigate a little more.

Help Center: to get (image) statistics

2008-08-28T05:04:39.0012Z

Thank you. It seems I don't know enough of MQL language; let me know if there's reference other than this website for I don't want to bother you like this.

I use the following code embedded in a perl code to count the number of topics with at least one image, and the result is about 438K--much lower than yours. By the way how to get the number of topics with exactly 1, 2, or 3 images? Can this be requested directly in the query?

{
"qname":{
"cursor":<cursor value>

    "query":[{
      "id":null,
      "limit":1000,
      "type":"/common/topic",
      "image":[{
        "id":null
      }]
    }]
}
}

Help Center: to get (image) statistics

2008-08-27T22:31:31.0012Z

You can just break the link between an image and a topic and not have the image deleted. Just for fun, I ran a quick analysis:

Found 716008 /common/topic topics with image links (out of ~4 million topics)
650326 have one img
63058 have two imgs
2296 have three imgs
328 have four or more imgs

Note that more than one topic can be linked to the same image.

Help Center: to get (image) statistics

2008-08-27T20:54:54.0012Z

Thank you. Then if a /common/image object is not linked to any /common/topic object, why is it there and how can it be presented to end users? Thanks.

Help Center: to get (image) statistics

2008-08-27T19:06:31.0012Z

No, not necessarily - a /common/image topic may not be linked to a /common/topic topic (for whatever reason). You could use the last query to find out how many images each /common/topic topic has by counting the ids in the "image" clause.

Computers: hidden properties on programming languages

2008-08-26T22:12:42.0012Z

can i also suggest a property on programming languages named "implementations" with an expected type of software?

Computers: Private Data

2008-08-20T09:22:44.0012Z

Thanks very much for your help on both counts.

Bug / Feature Request: Length of Auto Suggest List

2008-08-19T22:53:38.0012Z

Thank you so much!

Bug / Feature Request: Length of Auto Suggest List

2008-08-19T18:42:28.0012Z

This is underdocmented, but you can always enter an ID in the autocomplete box. So once you know that “Midnight” is /guid/9202a8c04000641f8000000008e684ab, you can enter that literally into the autocomplete box.

4. Metaweb Read Services: c# example

2008-08-19T17:42:53.0017Z

As there is not one above, i created a C# version here :

http://tinyurl.com/6y6hea

If anyone knows a good place to put it please do so. New to this so don't wanna edit the description of the article!

Bug / Feature Request: Length of Auto Suggest List

2008-08-19T17:33:05.0012Z

I've got a similar problem with similar symptoms - it seems at the very least, priority should go to exact name/type matches first, then exact name matches, then approximate matches. This isn't what happens and it causes all sorts of grief if the topic name is a simple word.

For example, I want to link to a Doctor Who TV Episode called "Midnight". If I type the word, the topic I want doesn't appear as an auto-suggest - many other topics mentioning the word appear first, some of which aren't even TV episodes. At this point I have only one choice, create a new topic, and plan to merge it later. Of course, the problem is, I can't submit a merge report either, since I get the exact same auto-suggest list with the entry I want too far down.

At the very least, would it be possible for me to use the guid? I can filter down and find the node from the full search box.

Computers: Private Data

2008-08-19T14:08:48.0013Z

Short answer: no. Freebase is a data commons. All data in it is publicly available. Some of it is harder to find than others (such as unpublished domains) but anything in Freebase can be found with the right query.

While the data in Freebase is licensed under a few different open data licenses, the software underlying the site is currently the proprietary property of Metaweb Technologies. Currently, we are focused exclusively on building the data commons; it is possible that private metawebs will be available at some point in the future, but for now, it’s all about the Freebase commons.

Computers: Private Data

2008-08-19T11:54:34.0017Z

I really love the Freebase data model. As well as contributing to the public data, I would also love to use it to log and organise my private data i.e. contacts, account numbers. Is it possible to create a domain that is viewable only to myself? Or alternatively, can the Freebase software be downloaded for personal use?

Computers: How do I create a new table (at whatever level is appropriate)

2008-08-14T22:51:12.0012Z

No problem, Mark. Freebase is quite powerful, but modeling new kinds of information is correspondingly difficult. We’ve tried to make the UI easy to use, and are still working on that, but there is an irreducible complexity, if you will, to the actual modeling process. Testing software seems like an interesting and useful domain to model, so please do let us know if we can help, if and when you do come back, maybe with a posse at your back. (-:

Computers: How do I create a new table (at whatever level is appropriate)

2008-08-14T21:39:23.0012Z

Sorry I tried to say thanks for taking the time to help - but that didn't get added.

Computers: How do I create a new table (at whatever level is appropriate)

2008-08-14T20:10:14.0000Z

Honestly this turns out to be way more complicated than I had hoped. At this stage I will just create a google spreadsheet.

Computers: How do I save a view?

2008-08-13T21:23:03.0000Z

You can't actually save a view, per se -- you can only save a filter. E.g., if you filtered the view of your type for Domain Specific Language = "yes", you could save that. However, there is currently a work-around (although I doubt this was intentional and I can't promise that it will be around forever) -- if you click the "filter results" button, a field labeled "save view as" will appear; you can enter a name and save the view without actually running a filter. The saved view will then appear on your homepage (I think), and will have a stable URL.

Computers: How do I save a view?

2008-08-13T21:18:11.0012Z

Replied to related post in the Computers domain.

Computers: How do I create a new table (at whatever level is appropriate)

2008-08-13T21:16:06.0005Z

You seem to have mistaken Freebase for a relational database. (-:

I note that your new type is full of text-valued properties that really should expect other types. Take a look at the Software type, for instance; the “Compatible Operating Systems” property expects a Software Compatibility type that links which links the OS and the versions in a compound value type (CVT).

You would probably benefit from checking out the Creating Types and Properties help pages, particularly the Data Modeling Guide. I think this will help answer questions about the views, as well.

Computers: hidden properties on programming languages

2008-08-13T21:07:51.0012Z

bump!

Computers: How do I save a view?

2008-08-13T20:53:49.0005Z

I've created a type: http://www.freebase.com/type/view/user/mlevison/agile_software_development/functional_test_tools and wish to change the properties that visitors will see in the default case (i.e no image, etc). How can I achieve that?

The whole goal of this is to build a repository of "Functional test tools" that users can quickly scan to figure out which tool they need.

As it stands the default view is of no interest.

Confused in Ottawa

Computers: How do I create a new table (at whatever level is appropriate)

2008-08-13T18:52:32.0005Z

I'm trying to document the wide array of Agile Functional test tools, in a searchable form.

My first attempt at this: http://functionaltestingtools.pbwiki.com/FrontPage is bit of mess.

Do I add the tools as individual topics and just tag them with types?

If someone could just point me to a concrete example: i.e. Blackberry is a topic. and its types are XXX.

If I've got this figured out correctly I need to add a topic per tool and them create a view. Which leads to the problem how do I create topics and views?

Looking on this page "Computer Domain" I can't see any add link for either topic or view.

Olympic Games: athletes by country: Beware of Long Lists!

2008-08-09T17:37:58.0000Z

Hmm, it seems like even though the import tool timed out, a lot of my topics were actually added. I still need to reconcile the whole list to find out which ones are missing though.

Olympic Games: athletes by country: Beware of Long Lists!

2008-08-09T01:08:09.0017Z

I just reconciled a list of 332 olympic athletes only to have the list import tool time out and loose all my work. If I can bring myself to do it all over again, I'll make sure to import it in batches of 100 but I thought I'd mention it because I wouldn't want anyone else to go through this.