tiflolinux.org - GNU Social
  • Login

Bienvenido

  • Public

    • Public
    • Network
    • Groups
    • Popular
    • People

Conversation

Notices

  1. jartigag (jartigag@mastodon.social)'s status on Sunday, 30-May-2021 21:00:02 CEST jartigag jartigag

    recently i started an analysis project, using #lichess data. these screenshots were just downloading this year so far and uncompressing a month 🥵

    now i have to preprocess such a BIG data 😅

    In conversation Sunday, 30-May-2021 21:00:02 CEST from mastodon.social permalink
    • jartigag (jartigag@mastodon.social)'s status on Sunday, 30-May-2021 21:00:11 CEST jartigag jartigag
      in reply to

      6h 😶 fortunately it was done in one night

      In conversation Sunday, 30-May-2021 21:00:11 CEST permalink
    • jartigag (jartigag@mastodon.social)'s status on Sunday, 30-May-2021 21:00:11 CEST jartigag jartigag
      in reply to

      such a simple "tweak" (much simpler than multiprocessing.Pool and anything else i've tried these days) make a decisive improvement in performance ✌️

      In conversation Sunday, 30-May-2021 21:00:11 CEST permalink
    • jartigag (jartigag@mastodon.social)'s status on Sunday, 30-May-2021 21:00:11 CEST jartigag jartigag
      in reply to

      from this (1% each ~3h):

      In conversation Sunday, 30-May-2021 21:00:11 CEST permalink
    • jartigag (jartigag@mastodon.social)'s status on Sunday, 30-May-2021 21:00:12 CEST jartigag jartigag
      in reply to

      found a solution. #grep to the rescue! 🦸

      https://github.com/jartigag/chess-blunders/blob/master/data/raw/pre_preprocess.sh

      In conversation Sunday, 30-May-2021 21:00:12 CEST permalink
    • Ekaitz Zárraga 👹 (ekaitz_zarraga@mastodon.social)'s status on Sunday, 30-May-2021 21:00:40 CEST Ekaitz Zárraga 👹 Ekaitz Zárraga 👹
      in reply to

      @jartigag let me tell you something: one of the best ways to reduce the processing time is to remove the prints :

      In conversation Sunday, 30-May-2021 21:00:40 CEST permalink
    • jartigag (jartigag@mastodon.social)'s status on Sunday, 30-May-2021 21:13:51 CEST jartigag jartigag
      • Ekaitz Zárraga 👹

      @ekaitz_zarraga yeah, i could remove them now that i have an acceptable preprocessing method 👌

      but look at the times in that case: i need at least one print each 3 hours to know that it keeps working! 😂

      In conversation Sunday, 30-May-2021 21:13:51 CEST permalink

Feeds

  • Activity Streams
  • RSS 2.0
  • Atom
  • Help
  • About
  • FAQ
  • TOS
  • Privacy
  • Source
  • Version
  • Contact

tiflolinux.org - GNU Social is a social network, courtesy of tiflolinux.org. It runs on GNU social, version 2.0.1-beta0, available under the GNU Affero General Public License.

Creative Commons Attribution 3.0 All tiflolinux.org - GNU Social content and data are available under the Creative Commons Attribution 3.0 license.