/
29/06/2022 Backend Meeting Minutes

29/06/2022 Backend Meeting Minutes

Owned by Varun Sethu
Jun 29, 2022
2 min read

Progress so far

  • Moving towards completion of concurrent editor, currently there is a barebones editor backend for the frontend MVP though

 

Discussion

  • The OT transformation functions outlined in the paper require numerical based indexing, unfortunately Brian’s functions rely on “string paths“, in fact any reasonable structure that we can serialize to disk will require string based paths.

  • We need a way of introducing numerical based indexing to our structure.

  • Ideas

    • We maintain 2 data-structures, our original big serializable JSON structure (as defined in https://github.com/csesoc/cms.csesoc.unsw.edu.au/blob/editor/Backend/editor/models/datamodel.go ) and a secondary tree structure that directly maps onto this datamodel structure

    • Each node of this tree-data structure will contain two values, the name of the field and its numerical indexing, we also modify Brian’s functions to work off numerical indexing.

    • Since Brian’s functions due to the nature of reflection will need to fetch the name of a field given its numerical index we need to introduce some index → field translation layer

      • First option is the introduce a caching layer, on a cache miss we traverse the structure and locate the name and pre-cache adjacent fields (the fundamental assumption here is that if an operation touches a field subsequent operations will touch adjacent fields)

        • There are some few complications as we need to introduce a precaching policy and a cache eviction policy

      • Second option is to maintain a huge version of the cache that translates everything, we need to update this table whenever there are any insert/delete operations that modify the JSON structure however doing this eliminates the need for a pre-caching and cache eviction policy.

      • We will probably go for the second option and benchmark memory usage in a more “productiony“ environment, if the memory usage is insane then we will move over to the caching idea.

  • Work Assigned

    • Transformation function implementation - Alex

    • Implementation of new translation layer - Gary

    • Modification of Brian’s functions to work with index based paths + General backend work to help frontend reach their MVP (implementing a publish endpoint , refactoring the filesystem table to use UUIDs)- Varun

    • Brian is a bit full with uni rn so taking it a little easy

 

Related content

UNSW CSESoc