35 coaches online • Server time: 00:46
* * * Did you know? The most aggressive player is Taku the Second with 6628 blocks.
Recent Forum Topics  How to run the FUMBB... | In-Season team build... | Dealing with a massi... |
2014-11-16 21:15:06
26 votes, rating 6
26 votes, rating 6
fumbblplus devnote #1
WARNING! Huge techical text!
In the past few weeks I managed to arrange many of my fragmented Python codes into a library called pyfumbbllib. I even coded some new stuff. I want to share what can be done with the library at the moment.
Currently the library have three components, bbcode, formats, and session.
bbcode
Currently it only has a table module I used to make the table below from a CSV file. New features will be added by my own needs for posting and editing of group informations.
formats
My prior goal was to mix the informations of the different sources of match results into a single object which can be used for further analysis or magic. Currently, the following formats are available for this:
With the formats module I am also capable of watch FUMBBL for new matches played and trigger events based on them.
Future plans:
session
With this module, I can automate tasks on FUMBBL. Currently I can log in and make/edit/delete a group or a note easily. Soon I will be able make alterations in them based on events triggered by the formats module.
This very blog post was tested by just editing a TXT file and updating a note with a plain command.
Future plans:
I am planning to run my script suite on a Raspberry Pi 24/7 and manage and provide some great stuff with it.
UPDATE#1: session module went private
In the past few weeks I managed to arrange many of my fragmented Python codes into a library called pyfumbbllib. I even coded some new stuff. I want to share what can be done with the library at the moment.
Currently the library have three components, bbcode, formats, and session.
bbcode
Currently it only has a table module I used to make the table below from a CSV file. New features will be added by my own needs for posting and editing of group informations.
formats
My prior goal was to mix the informations of the different sources of match results into a single object which can be used for further analysis or magic. Currently, the following formats are available for this:
- HTML: The HTML match report we all know well. This one gets used as a basis of the detailed match report.
- API-XML: The XML provided by the FUMBBL API, which is the only source of information about the division_id which is essential. Unfortunately, it is not provided for all matches (BUG #1868), though no problem with recent ones.
- FFB-XML: The XML which is probably used by the FFB server to update the database and is stored on the site. Unfortunately, this one is also missing for many matches (BUG #1878), but again no problem with recent ones.
- FFB replay data: This is the ultimate source which contains all aspects of a match. Fortunately it is currently migrating to a human readable form which seems to JSON for the first view but I had not downloaded any data yet.
Key | HTML | API-XML | FFB-XML |
| division_id | X | ||
| gate | X | X | x |
| match_id | x | X | |
| replay_id | X | X | |
| replay_version | X | ||
| team.FAME | X | X | |
| team.TR | X | ||
| team.TS | X | ||
| team.TV | X | X | X |
| team.TW | X | ||
| team.coach_id | X | ||
| team.coach_name | X | ||
| team.coach_postmatch_CR | X | ||
| team.coach_postmatch_bracket | X | ||
| team.coach_rankings | X | ||
| team.conceded | X | X | X |
| team.fanfactor_delta | X | X | X |
| team.inducements | X | X | |
| team.name | X | X | |
| team.petty_cash_transferred | X | ||
| team.petty_cash_used | X | X | |
| team.player.defecting (?) | X | ||
| team.player.injuries | X | X | |
| team.player.name | X | ||
| team.player.nr | X | ||
| team.player.origin | X | ||
| team.player.player_id | X | X | |
| team.player.player_type | X | ||
| team.player.spps | X | ||
| team.player.stats | X | X | X |
| team.player.stats.spps | X | X | |
| team.race | X | ||
| team.score | X | X | X |
| team.spectators | X | X | |
| team.spiralling_expenses | X | X | |
| team.stats | X | x | x |
| team.stats.cas_bh | X | X | |
| team.stats.cas_si | X | X | |
| team.stats.cas_kill | X | X | |
| team.stats.suf_cas_bh | X | ||
| team.stats.suf_cas_si | X | ||
| team.stats.suf_cas_kill | X | ||
| team.team_id | X | X | X |
| team.winnings | X | X | X |
| time | X | X | |
| type | X |
With the formats module I am also capable of watch FUMBBL for new matches played and trigger events based on them.
Future plans:
- add remining formats
- deal with current games page
- add remaining API formats for groups, tournaments, etc.
session
With this module, I can automate tasks on FUMBBL. Currently I can log in and make/edit/delete a group or a note easily. Soon I will be able make alterations in them based on events triggered by the formats module.
This very blog post was tested by just editing a TXT file and updating a note with a plain command.
Future plans:
- tournament automation
- message editing
- wiki editing
I am planning to run my script suite on a Raspberry Pi 24/7 and manage and provide some great stuff with it.
UPDATE#1: session module went private
Comments
Posted by Dunenzed on 2014-11-16 22:52:57
I rated this 6, even though I understood very little. Match reports getting further analysis or magic sounds fantastic!
Posted by pythrr on 2014-11-17 00:08:54
who are you who is so wise in the ways of wiz-hackery?
Posted by PurpleChest on 2014-11-17 00:48:07
none of that made any sense to me whatever. Maybe a version written for the less intelligent sometime?
i still couldn't define what you are trying to do, let alone how. But good luck with whatever it is you are trying to do.
i still couldn't define what you are trying to do, let alone how. But good luck with whatever it is you are trying to do.
Posted by Lorebass on 2014-11-17 03:10:25
Yeah, having to understand what that meant is fine at work... but not at no-thinky-home-time.
could use an executive summary
could use an executive summary
Posted by member1234 on 2014-11-17 03:19:45
So pulling info from fumbbl togther and enetering/editing it via a 3rd party app? Live updates and stats etc. as new data is grabbed? (which can then be auto updated to tourney/league/group/profile pages) Something akin to a fumbbl app? (Or those sites that grab xbox achievements from live, to use one thing vaguely similar I know of).
So you could, say, build an achievements/badge system based on all the data that can be grabbed, or work through any (live) stats or queries you feel like or auto updating top list stats etc.
Over my head too really, more magic sounds good!
So you could, say, build an achievements/badge system based on all the data that can be grabbed, or work through any (live) stats or queries you feel like or auto updating top list stats etc.
Over my head too really, more magic sounds good!
Posted by happygrue on 2014-11-17 03:20:00
If I am reading you right, this would let you... mine matches for data via some other script, allowing something like "which player in the meta group ARR scored the most cas in the past two months", and such other awesomeness? Even if not that some some lesser semi-automated match mining would be awesome. Keep up the good work!
Posted by SzieberthAdam on 2014-11-17 07:42:18
@member1234 and happygrue: Exactly.
Posted by roante on 2014-11-17 08:10:29
Well done, mate!
Posted by Balle2000 on 2014-11-17 10:10:07
Rated 6 'cause spider sense went: this gonna rock
Posted by Wreckage on 2014-11-17 12:43:13
My only thoughts were... why would you steal 80GB of data and did Christer condone this and wth did you do with it?
Posted by Kam on 2014-11-17 12:45:49
How can you "steal" data freely accessible on the internet?
Posted by SzieberthAdam on 2014-11-17 14:19:52
@Wreckage: I needed the data locally for testing my code. The current version of format parser was tested on over 300k match reports. Trust me, even after you test 200k you will face an exception on the next one. Some of the exceptions were raised because the bugs in my code, some were raised because of FUMBBL bugs, which I had to handle and try to get the best of them.
I hope Christer welcomes my handful of bug reports. :)
It also allows me to provide all-time/annual stats without the need to get all match reports from the server again and again.
I hope Christer welcomes my handful of bug reports. :)
It also allows me to provide all-time/annual stats without the need to get all match reports from the server again and again.
Posted by Christer on 2014-11-17 15:13:21
This is pretty cool. You'll be happy to hear that I've begun work on yet another version of the API (which will clearly be better than sliced bread :) ).
I took a quick look at one of your bugs (missing data from a match result through the XML). The quick answer is that the ID you're querying for is one of the two match results that the old JavaBBowl client uploaded. The one that you will get XML data for is one number lower. Mind you, this was very early days, so it's quite likely that there are bugs..
Anyway, I'd be happy to talk about the API with you in more detail, and more specifically I'd be happy to get some input into what data I could include in various API calls.
As for the ownership and "legality" of this, it's not a big deal. The information downloaded is public and in no way secret. The APIs are there for this kind of thing and I have no issue with people using the information for various analytical purposes.
However, the session stuff poses a bit of a problem in some use cases. It would, for example, be possible to make an external site that allows people to add credentials (for example for automatic addition of stats tables into team bios). This would, though, be against the "accounts are personal" policy and is not allowed in this form. Basically, if you can't do it through the XML APIs, it's not allowed or supported. That being said, I don't mind it for testing purposes.
Now, the new API I'm working on has planned support for token based OAUTH, which would allow me to do things like this in a more secure fashion. Nothing actually implemented at this point, but at least it's planned. Once we're past the FFB client update, I'll go back to implementing the Road to the NAFC tourney options and the parts of custom league rules that are related. These features are currently utilizing the new API format I'm working on, and I've also talked to Kalimar about potentially migrating the existing XML API use to this new one (which incidentally supports both XML and JSON).
Anyway, It'd be great to have a client API toolkit available for FUMBBL and I'd happily assist in making it as good as it can be. I wouldn't mind having a work group of sorts to discuss it once I free up a bit of time to flesh out the new API :)
I took a quick look at one of your bugs (missing data from a match result through the XML). The quick answer is that the ID you're querying for is one of the two match results that the old JavaBBowl client uploaded. The one that you will get XML data for is one number lower. Mind you, this was very early days, so it's quite likely that there are bugs..
Anyway, I'd be happy to talk about the API with you in more detail, and more specifically I'd be happy to get some input into what data I could include in various API calls.
As for the ownership and "legality" of this, it's not a big deal. The information downloaded is public and in no way secret. The APIs are there for this kind of thing and I have no issue with people using the information for various analytical purposes.
However, the session stuff poses a bit of a problem in some use cases. It would, for example, be possible to make an external site that allows people to add credentials (for example for automatic addition of stats tables into team bios). This would, though, be against the "accounts are personal" policy and is not allowed in this form. Basically, if you can't do it through the XML APIs, it's not allowed or supported. That being said, I don't mind it for testing purposes.
Now, the new API I'm working on has planned support for token based OAUTH, which would allow me to do things like this in a more secure fashion. Nothing actually implemented at this point, but at least it's planned. Once we're past the FFB client update, I'll go back to implementing the Road to the NAFC tourney options and the parts of custom league rules that are related. These features are currently utilizing the new API format I'm working on, and I've also talked to Kalimar about potentially migrating the existing XML API use to this new one (which incidentally supports both XML and JSON).
Anyway, It'd be great to have a client API toolkit available for FUMBBL and I'd happily assist in making it as good as it can be. I wouldn't mind having a work group of sorts to discuss it once I free up a bit of time to flesh out the new API :)
Posted by SzieberthAdam on 2014-11-17 16:29:50
I am glad you like it Christer! Soon I will share my suggestions on the Deveoper forum.
I deleted the current repository. The session module will be private from now on. If someone is interested please contact with me. The rest of the modules will be available again soon on the same place.
I deleted the current repository. The session module will be private from now on. If someone is interested please contact with me. The rest of the modules will be available again soon on the same place.
Posted by akaRenton on 2014-11-17 23:54:06
Witchcraft!
Posted by Stimme on 2014-11-18 02:44:29
Nice! And it looks like it is only a question of time, until Zawinski's law is fulfilled.