Not signed in (Sign In)

Auctioneer::Technical: Auc-Stat-Purchased treats cancellations as purchases

Bottom of Page

1 to 48 of 48

  1.  
    • CommentAuthorNine
    • CommentTimeDec 10th 2007 edited
     
    I noticed this when I canceled some of my own auctions because I'd been undercut by 1c. :) I don't think the scanner detects the player's cancellations at the moment, which might be a useful thing to add, but in the general case we can't reliably tell if an auction has been canceled or bought.

    How about this logic though

    if an auction has disappeared then
    if pre-existing auctions at the same (or similar?) quantity and lower price have NOT disappeared then
    it was probably canceled, ignore it
    else
    it was probably bought, do stuff
    end
    end

    There's a chance of false positives here but I'd rather have a small amount of good data than a large amount of bad data. :)
    • CommentAuthorRockSlice
    • CommentTimeDec 10th 2007
     
    The trouble with this is that it would require a search through the scandb for every single missed item (which can easily number in the thousands)

    The commit code already causes a noticeable lag at the end of a scan.
    • CommentAuthorNine
    • CommentTimeDec 10th 2007 edited
     
    Yeah, I can see it causing some slowdown, but I don't think it requires hitting the scan db - aren't the stats modules already given a full list of unchanged/updated auctions during the snap update? Purchased could store those for cross reference with missed auctions and discard the data afterwards. I'll look into it tonight. :)
    • CommentAuthorNine
    • CommentTimeDec 11th 2007
     
    So I've created another stats module based on purchased so I can run the two in parallel and compare.

    Re slowdown, I was wondering, has anyone tried using Lua coroutines to offload heavy operations like this onto a "background thread"?
    • CommentAuthorRockSlice
    • CommentTimeDec 11th 2007
     
    I've done a bit of dabbling, mainly to troubleshoot why people were getting DCed by the GetAll
    • CommentAuthorKinesia
    • CommentTimeDec 12th 2007
     
    I've just started playing around with Purchased and noticed that even when it has data it won't use it as a default pricing scheme. (I tried used Purchased as the first default and Statdev as the second)
    Even when I can see the data in the tooltip.
    I doubt it's anything to do with this but thought it served mentioning.
    I opened up a ticket for it at the time I noticed.
    • CommentAuthorKinesia
    • CommentTimeDec 16th 2007
     
    This will now work as a pricing scheme in Appraiser.
    It's not entirely useful as one yet though, it needs access to the Beancounter information so it can learn a bit about Bid prices as well and get more accurate buyout figures for your own auctions, then it'll be more useful since it will better show what you last sold things for and stuff like that.
    •  
      CommentAuthordinesh
    • CommentTimeDec 16th 2007
     
    i am very conflicting about promoting Purchased stat too much, as in my experience it is not very predictive and is often way overpriced. of course, I welcome attempts to improve it.

    I do think there is room for a stat which collects actual sales prices from you, others in your guild, and/or others on the server. this is something we talked a lot about early on in AA dev days, but not so much recently.
    • CommentAuthorNine
    • CommentTimeDec 16th 2007
     
    As well as the cancellation thing, there's some bugs in query matching that make it unreliable, with items being removed from the scan image inappropriately. I see one of them in JIRA (http://jira.norganna.org/browse/ADV-51) marked as working as intended. I have a fix for it but it's a little slow because it requires tooltip scanning - there doesn't seem to be any other way to find out whether an item would match with "Usable Items" checked.
    • CommentAuthorNine
    • CommentTimeDec 16th 2007 edited
     
    My experimental version with cancelation checks is here: http://norganna.org/up/show.php/vmK1XZ/Auc-Stat-Purchased2.zip
    • CommentAuthorKinesia
    • CommentTimeDec 16th 2007
     
    I fixed it because I wanted to play around with having Purchased as my default pricing scheme and Market as my second, in the hope it'd reflect what things were really being sold at, but as you say, in it's current form it's not useful (Certainly not in the way I was trying to use it).
    I find it really weird that you think it's often overpriced... I almost always found it way under Market Price, to the extent that I really didn't want to sell things at that price. (Hence my thought that the Beancounter information would help.) I actually played around with letting Market-Matching go 100% over purchased price to match the market and even _that_ wasn't enough to get it to around market price normally. (And it was very bad with Auctions that had no competition)
    So I agree that at the moment it shouldn't be promoted. Collecting the stats is important, but how to actually use them is another matter... (Although I think it's influence on Market price probably remains useful.)
    We were talking about it a bit in IRC on the weekend and Beancounter support isn't too difficult to add, but optimising it so it doesn't slow things down may be quite another matter.
    It makes me think that one of the other suggestions may be the best way to go... A Purchased-Matching algorithm that'll skew your other figures towards the Purchased price but not actually use it as the base.

    Thanks Nine, I'll have a look at your version later.
    • CommentAuthorNine
    • CommentTimeDec 16th 2007
     
    Thanks, lemme know what you think. I just updated the link with a new version. Couple of minor bugfixes and some improvements to the coroutine job scheduler.

    Is it ok to stick stuff like this in SVN? Wouldn't want it to show up in releases but since it's a separate addon I assume it won't unless it's added to the build scripts.
    •  
      CommentAuthordinesh
    • CommentTimeDec 17th 2007
     
    It is much preferable to ask this sort of thing in the dev channel in IRC I think. My general philosophy is that one can always check something into SVN, though at times you might want to talk about the design with the project lead (i.e. MP) first if it's a major change or if it is likely to be somehow controversial. Plus, as you say, until it's added to build.conf, it won't get added to the release packages.
    • CommentAuthorKinesia
    • CommentTimeDec 17th 2007
     
    One possible way to implement a matching module using purchased would be along the lines of:
    "Undercut the lowest auction that is priced between Market Price and Purchased Price."
    • CommentAuthorNine
    • CommentTimeDec 19th 2007
     
    Or getting fancier, auto-undercut: undercut by sufficient amount to give x% chance of sale, based on past sales history. For this you'd need to remember the undercut amount used for each auction posted.
    • CommentAuthorKinesia
    • CommentTimeDec 29th 2007
     
    I like your version Nine, the numbers it gives back are pretty good, but there seems to be a bug...

    Date: 2007-12-29 23:22:49
    ID: 52
    Error occured in: Global
    Count: 1
    Message: ..\AddOns\Auc-Advanced\CoreAPI.lua line 92:
    attempt to compare number with string
    Debug:
    (tail call): ?
    [C]: ?
    Auc-Advanced\CoreAPI.lua:92: GetMarketValue()
    Auc-Util-PriceLevel\PriceLevel.lua:302: CalcLevel()
    Auc-Util-CompactUI\CompactUI.lua:542: RetrievePage()
    Auc-Util-CompactUI\CompactUI.lua:684:
    Auc-Util-CompactUI\CompactUI.lua:656
    [C]: ?
    [C]: AuctionFrameBrowse_Update()
    ...ace\AddOns\Blizzard_AuctionUI\Blizzard_AuctionUI.lua:382: AuctionFrameBrowse_OnEvent()
    [string "*:OnEvent"]:1:
    [string "*:OnEvent"]:1


    For some items it ends up returning "-1.$" as the price which doesn't make much sense. I assume it's a problem with pushing something through "%0.01f" because there doesn't seem to be anywhere else for it to come from.
    This is what a few lines in my SV look like:
    [3196] = "1183:1;1;8068.0;8068.0;8068.0,1551:1;0;-1.$;-1.$;-1.$,589:1;0;-1.$;-1.$;-1.$,97:1;0;-1.$;-1.$;-1.$,503:1;1;5508.0;5508.0;5508.0,1015:1;0;-1.$;-1.$;-1.$,674:1;0;-1.$;-1.$;-1.$,93:1;1;4495.0;4495.0;4495.0",
    • CommentAuthorKinesia
    • CommentTimeDec 29th 2007
     
    It appears to _always_ be " of the " items if that helps.
    • CommentAuthorRockSlice
    • CommentTimeDec 29th 2007
     
    Posted By: NineOr getting fancier, auto-undercut: undercut by sufficient amount to give x% chance of sale, based on past sales history. For this you'd need to remember the undercut amount used for each auction posted.


    The BeanCount Matcher will do this. It doesn't do anything fancy like remembering the prices used before, but simply checks the number of succeeded/failed auctions of that item, and increases or decreases the price accordingly.

    To achieve a specific % chance of sale, you'll need to do some math, though.

    EG:
    for 50% chance of sale, set (increase on success) = (decrease on failure)

    for 67% chance of sale, set (2*increase) = (decrease). For prices to stabilize, you need 2 successes, and 1 failure.
    • CommentAuthorNine
    • CommentTimeJan 2nd 2008
     
    Posted By: KinesiaIt appears to _always_ be " of the " items if that helps.


    I think those are NaNs - I see a few on mine too. Something up with property handling I guess. I'll take a look.
    •  
      CommentAuthorkandoko
    • CommentTimeJan 2nd 2008
     
    Nine, you may have the same issue I had with beanCounter. GetItemInfo(itemId) will strip "of the XXX" off, it returns the bare itemlink
    So |cff1eff00|Hitem:4569:0:0:0:0:0:6:-1760802159|h[Staunch Hammer of Strength]|h|r becomes
    |cff1eff00|Hitem:4569:0:0:0:0:0:0:0|h[Staunch Hammer]|h|r
    and looses all the semi random stats.
    • CommentAuthorNine
    • CommentTimeMar 17th 2008 edited
     
    So I'm still working on this, it's just a bit more complicated now. :)

    One little niggle I've run into though is an artifact of the way the scanner works. When an item is first "missed" it's still passed through to the stats modules. Only the second time it's missed is it actually removed. What can happen with this is

    Scan 1: Auction for 1g is missed, new auction for 95s is seen and added
    Scan 2: Auction for 1g is missed again and removed

    Because the 95s auction was added before the 1g auction was removed, the 1g auction looks like a cancellation, even though it had most likely already been bought when it was missed first time around.

    So what I'm wondering is, would there be way for the scanner to tell stats modules when an auction was first/last seen so that I can just compare the times? It doesn't look like it tracks that information at the moment.
    • CommentAuthorNine
    • CommentTimeMar 21st 2008 edited
     
    Posted By: NineSo what I'm wondering is, would there be way for the scanner to tell stats modules when an auction was first/last seen so that I can just compare the times? It doesn't look like it tracks that information at the moment.


    I found a way around this by remembering first seen time in the stats module. This required a change to corescan which currently doesn't give auctions an id until AFTER they've first been passed to stats modules. (There's no way other than id to uniquely identify an auction).

    Auction id seems like a rather "internal" thing to be depending on though; it doesn't allow stats data to be shared across accounts for example, because ids will be different. Anyone know what would be the implications of adding an extra field to scandata to track first seen?
    • CommentAuthorKinesia
    • CommentTimeMar 26th 2008
     
    Nine, can you give it your own id in the stats module and keep track that way?

    If you have a copy we can play around with to test that'd be cool...
    • CommentAuthorNine
    • CommentTimeMar 27th 2008 edited
     
    http://www.norganna.org/up/show.php/nEX5U0/Auc-Stat-Purchased2.zip

    This is kinda old but it should be stable. I've moved Jobby into its own addon now. Assigning ids inside stats would add a lot of complexity. Stats modules have no knowledge of the current query or page index/count which the scanner can use to uniquely identify auctions. The scanner change was

    Index: Auc-Advanced/CoreScan.lua
    ===================================================================
    --- Auc-Advanced/CoreScan.lua (revision 2972)
    +++ Auc-Advanced/CoreScan.lua (working copy)
    @@ -417,7 +417,7 @@

    local statItem = {}
    local statItemOld = {}
    -local function processStats(operation, curItem, oldItem)
    +local function processStats(operation, curItem, oldItem, idList)
    if (curItem) then private.Unpack(curItem, statItem) end
    if (oldItem) then private.Unpack(oldItem, statItemOld) end
    if (operation == "create") then
    @@ -437,6 +437,11 @@
    end
    end
    end
    + -- If this is a new auction and not being filtered, assign it an id now
    + if (operation == "create" and idList) then
    + curItem[Const.ID] = private.GetNextID(idList)
    + statItem.id = curItem[Const.ID]
    + end
    elseif curItem and bit.band(curItem[Const.FLAG] or 0, Const.FLAG_FILTER) == Const.FLAG_FILTER then
    -- This item is a filtered item
    operation = "filter"
    @@ -633,8 +638,9 @@
    end
    scandata.image[itemPos] = clone(data)
    else
    - if (processStats("create", data)) then
    - data[Const.ID] = private.GetNextID(idList)
    + if (processStats("create", data, nil, idList)) then
    + -- (nine) moved this to processStats, so that new auctions get an id BEFORE being passed to stats modules
    + -- data[Const.ID] = private.GetNextID(idList)
    table.insert(scandata.image, clone(data))
    newCount = newCount + 1
    end

    Ideally, though, a "first seen" time in the scanner would be better. Tracking it in the stats module accounts for about half my stored data, despite normalizing it, when it would be one extra field in scandata.
    • CommentAuthorKinesia
    • CommentTimeMar 28th 2008
     
    Heh. My copy of CoreScan.lua now incorporates hacks from you, Mikk and Norgs...
    • CommentAuthorKinesia
    • CommentTimeMar 29th 2008
     
    I'm really liking your version so far Nine... I'd talk to MentalPower about the core change you want and get the release very of Purchased updated because it really needs some love...
    • CommentAuthorNine
    • CommentTimeApr 10th 2008 edited
     
    Cool, glad you like it. Lemme play with it some more though. For one thing, the data format is incompatible with the original -Purchased, because to match -Simple it tracks by volume rather than auction count. Latest here: http://up.norganna.org/index.php/lrESbSP/Auc-Stat-Purchased2.zip

    Three new things:

    * Jobby is in its own addon and schedules more intelligently
    * Added some profiling to Jobby to measure coroutine overhead: /run Jobby.private.profile() (I'd be interested in results from Windows machines as I run on Mac)
    * Added some seller stats analysis, should be interesting if you've been running this or the version above for a while: /aadv stat purchased2 sellers
    • CommentAuthorMikk
    • CommentTimeApr 10th 2008
     
    There's another problem with current scanning that your stat modifications partially adress.

    If you're running BTM while scanning, the following happens.

    * You view a page with an item on that you buy.
    * The first item from the next page gets scrolled into the page you are viewing. SERVER SIDE. But you never refresh locally.
    * You continue to the next page, and never see that item that got scrolled away.

    This is VERY obvious if you run a scan where you e.g. buy a bunch cloths or dusts or something like that. Re-run the scan again and whoopie suddenly there's even more of the SAME items to be bought that you didn't see the first time.

    The solution/fixing of this does not belong in this thread, but I thought I'd let you know that this is a reason for bad stats as well as cancellations. Possibly even the more common one of the two.
    • CommentAuthorNine
    • CommentTimeApr 11th 2008 edited
     
    This causes neither false removals nor false additions though, at least not often. Unless you're running a getall scan or the query results fit on one page (which is kinda the same thing) items need to be missed TWICE before they are removed from the image.

    One problem is that if you catch items on a second pass that you missed first time around, these aren't treated as new auctions if they match the ones you just bought. This means you underestimate both the number of additions and the number of removals, but the two balance out by definition (otherwise the image would grow indefinitely or shrink to nothing).

    You are going to miss plenty of auctions under normal conditions anyway, even if you leave Auctioneer running full scans 24/7. :)
    • CommentAuthorKinesia
    • CommentTimeApr 13th 2008
     
    Ouch, just found a severe error... (Don't know if it's present in the original version because it doesn't have the reporting!)
    Scanning at a neutral AH... And now it thinks everything at the main one was bought out...
    •  
      CommentAuthordinesh
    • CommentTimeApr 13th 2008
     
    definitely don't forget to log this in JIRA, if it's not there already.
    • CommentAuthorKinesia
    • CommentTimeApr 13th 2008
     
    Unfortunately I currently only know it's in Nine's version which is fairly modified, but since he was building off the original it remains fairly possible it's there as well...
    I'll try and look into it tomorrow to confirm one way or another.
    • CommentAuthorNine
    • CommentTimeApr 13th 2008
     
    I very rarely use the neutral AH so that's entirely possible, I'll take a look. A while ago I did notice that the manual "push" command wasn't working properly because the faction tracking was messed up, but that bug should be fixed. It's possible this is related.
    • CommentAuthorNine
    • CommentTimeApr 13th 2008 edited
     
    Hmm, it seems fine here. I flew down to Gadgetzan and realm/faction came through correctly as <myserver>/neutral, then back to <myserver>/<myfaction> when I returned to my usual AH, and only "real" purchases that had happened in the meantime were seen.

    ScanProcessors aren't passed realm/faction directly and need to depend on the global GetRealmName() and AucAdvanced.GetFactionGroup() - is it possible the latter is returning inappropriate values sometimes?
    •  
      CommentAuthorkandoko
    • CommentTimeApr 13th 2008
     
    Does the force faction data everywhere affect what it think the neutral AH is?
    • CommentAuthorKinesia
    • CommentTimeApr 13th 2008
     
    Did you scan while you were at Gadgetzan, Nine?

    Hmmm, Kandoko I do have that set...
    • CommentAuthorNine
    • CommentTimeApr 13th 2008
     
    Yeah I scanned twice. That catches new items and deals with things that were missed on the first scan.
    • CommentAuthorNine
    • CommentTimeApr 13th 2008 edited
     
    I used this to take a look. Paste it in with the comments uncommented to see what the stats modules are seeing

    function private.ValidateFactionAndRealm(faction, realm)
    --print("ValidateFactionAndRealm got "..(faction or "nil")..","..(realm or "nil"))
    if (not faction) or (faction == AucAdvanced.GetFaction()) then
    faction = AucAdvanced.GetFactionGroup()
    --print("ValidateFactionAndRealm adjusting faction to "..(faction or "nil"))
    end
    realm = realm or GetRealmName()
    --print("ValidateFactionAndRealm returning "..(faction or "nil")..","..(realm or "nil"))
    return faction, realm
    end

    (I added this function because I was sometimes getting "SomeRealm-SomeFaction" instead of just "SomeFaction" for the faction name...)
    • CommentAuthorNine
    • CommentTimeApr 14th 2008 edited
     
    Meh, this is why I'm so anal about APIs. No more inputs than are necessary and all valid. No global state. Shouldn't be error-checking and -correcting unless it's raw user input.
    • CommentAuthorKinesia
    • CommentTimeApr 14th 2008
     
    Yes, this setting is the problem.
    It's likely affecting all stats in this way because they can no longer ever tell they are at a neutral AH...

    Basically the code is meant to be checking whether you have the AH open or not and, if so, return the Neutral zone. If you are in a neutral zone without the AH open it will show home faction data. But it appears the check for whether you are at the AH doesn't work because it always returns the home faction.


    Added http://jira.norganna.org/browse/ADV-218
    • CommentAuthorNine
    • CommentTimeApr 14th 2008 edited
     
    I'm thinking scanprocessors should be passed a realm and faction (an accurate one :), at least on begin()? Scanprocessors can't be started except at the AH right now, but I don't see a fundamental reason why they wouldn't be kicked off elsewhere based on expiry time etc.
    • CommentAuthorKinesia
    • CommentTimeApr 14th 2008
     
    Ok, the issue seems to be the caching of the locale to faction mapping in the AucAdvanced settings.
    In mine (at least) Gadgetzan is set to Alliance which is obviously wrong.
    It works fine when removing the caching.
    • CommentAuthorKinesia
    • CommentTimeApr 14th 2008
     
    As far as I can see the code is fine, it's just bad data and we don't know how it's getting in yet... (Possibly from an earlier bug that's been fixed.)
    For now I did:
    /script AucAdvancedConfig.factions = {}

    to clear the bad data and we'll see if the problem comes back...
    • CommentAuthorNine
    • CommentTimeApr 14th 2008
     
    Posted By: Kinesiawe'll see if the problem comes back


    Nooooo!

    I know I come across as a twat because I work on my own and send random forum posts about stuff, and I am not the one ultimately fixing a lot of problems like this. But such bugs come about as a result of a large system depending on shared state. It happened with Auc Classic, which was an awesome piece of work which grew beyond its means, and broke, and now I see the same happening with Advanced.

    Of the new systems, I think ScanProcessor is particularly well-designed, but this current problem is a fundamental bug in its API that should not be worked around: stats modules need to know where their data is coming from. Having to call GetFactionGroup should not be necessary inside a stats module. They should be fed that information.
    • CommentAuthorKinesia
    • CommentTimeApr 14th 2008
     
    Heh. I don't disagree with you, but you should jump on IRC and talk MentalPower into it.

    It's actually the thing I _really_ dislike about Blizzards "api's" and a lot of the issues we have are coming directly from their implementation...
    This is an example itself... A lot of the problem with Faction, the reason we have to have an internal function to look it up, is because Blizzard's api function isn't consistent. If you run it when you first log on it returns nil. Which is rubbish. You have a faction as soon as you log in, but the Blizzard api doesn't know it until it queries the server and at login there are a big queue of those queries.
    In reality the loading screen should stay up a second longer to make sure that API calls are reproducable and always return their expected results regardless of client state.
    It's exactly the same as the seller names in the Auction house... The Blizzard API is messed up and so developers have to do weird workarounds to get what should be default behaviour...

    In this particular case... There's no bug for me to fix as far as I can see, just corrupt data... I've no way to know the existing code is causing corrupt data because the files are so old...

    The reengineering you want to do to distance the problem from the Scanners is a good idea, but it won't affect this problem because we'll still be running this same function in the corescan function and passing the same value in to each Scanner, so it's really just shuffling things around. (It's a good idea though because that code really need not be duplicated everywhere.)
    •  
      CommentAuthorHirsute
    • CommentTimeApr 14th 2008
     
    Kinesia: The corrupt data couldn't be sneaking in because of your heavily modified CoreScan.lua, could it?
    • CommentAuthorKinesia
    • CommentTimeApr 14th 2008
     
    Nah, it's only in CoreUtil.lua and fairly selfcontained.
    It seems to be profile based, so using the same profile on horde/alliance has probably messed up a user Nechkn was telling me about.
    For me, though, it claims Tanaris is alliance which it isn't for either faction, so there was a bug at some point confusing it...

    MP says it doesn't need to save between sessions though so I'll make the cache local and that should get rid of it...
    • CommentAuthorKinesia
    • CommentTimeMay 8th 2008
     
    Nine: Got a challenge for you...

    To do some future "tricks" we really need to know the prices things _aren't_ selling at (as well as the prices they do sell at). Theoretically the optimum price lies between the two. (Though of course the samples will overlap occasionally because some people pay more)

    I was thinking that Purchased already does most of the work.
    We obviously can't pick up everything that doesn't sell, any more than we can catch everything that does sell, but I was thinking...
    Anything that has gotten to it's last 30 minutes without a bid on it should count as a failed sale.

    I was thinking this could be returned in the tooltip (optionally) and by the GetPrice functions, but not alter the value returned the Market price and the rest for now, just use the additional optional fields.

    What do you think?

1 to 48 of 48

  1.  
World of Warcraft™ and Blizzard Entertainment™ are trademarks or registered trademarks of Blizzard Entertainment, Inc. in the U.S. and/or other countries.