Survey Matt Akamatsu's work around using Discourse Graphs for research #8

New Issue

ronent · 2023-07-17T06:05:40-05:00

ronentk commented

2023-07-17 06:05:40 -05:00

(Migrated from gitlab.com)

Related refs:

Roam project presentation page

Related refs: - [Roam project presentation page](https://roamresearch.com/#/app/akamatsulab/page/ud1vxU56v)

ronentk commented

2023-07-17 06:05:56 -05:00

(Migrated from gitlab.com)

marked this issue as related to #6

ronentk commented

2023-07-18 02:00:26 -05:00

(Migrated from gitlab.com)

Matt shared with me this excellent presentation (slides + video) he recently gave at the Allen Institute.
From what I can see, his lab is probably the most advanced in terms of actually using DGs for research - they haven’t actually written a paper directly using discourse graphs, but he runs the lab based on a shared collaborative DG, which forms the knowledge base underlying their research: (1) synthesizing existing knowledge (2) they also want to extend this to ongoing research
Some interesting points that came up (well worth watching in full as well):
- Matt mentioned a few times that UX is key, which jives with our intuitions as well. They also have a neat graphical UI to add nodes and connect them, a custom built add on to Roam that a developer created for them.
- Not too surprisingly, it seems like there are some nice synergies between DGs and LLMs - they experimented with LLMs summarizing parts of the graph, writing results sections based on the graph, etc. Another interesting area could be searching the graph using natural language queries over LLMs (”claims we’ve made related to gene ”).
- Matt mentions an NSF grant on using DGs for molecular sciences knowledge synthesis.
- Modular credit attribution is one of the benefits of using DGs. Matt also mentions that conveying research in the form of a shared DG (vs just research papers + code for example) could be made as a precondition to getting funding.
- This bulletin board (around min 31:00) is a cool example of how the data can be aggregated both as a leaderboard and a way to help orient new researchers - seems related to some of the convos around topics at DGF, and how to organize/present forum data.
- In general, orientation over the graph is a challenge, the dashboards partially address it, but there is still a lot of work to be done on this front including using AI.
Some takeaways for DGF
- This talk made me wonder again about what tooling/infrastructure to use for our experiment
  - They primarily use the Roam DG extension to create and navigate the graphs.
  - They use a Zotero - Roam integration to manage their references. Reference management seems like an important topic to consider. Maybe we could also use Zotero? Notion also works for this (and has a Zotero integration).
  - Re-raising the question regarding the forum custom graph UI - on one hand it will let us do new things that may be hard to add to Roam, but on the other hand it may take a while to get there, and usage friction will be high initially. Maybe we could try to use Roam or another similar tool initially and transition to our UI as it develops?
- Some contributions / perspectives DGF can bring to the table
  - Social/reputation incentives are a direction they still haven’t looked into much yet, though I see this being discussed with Protocol Labs here: https://research.protocol.ai/blog/2023/discourse-graphs-and-the-future-of-science/.
    - Social incentives - even basic feedback in the form of likes over graph nodes, etc.
    - Reputation - ways to summarize member contributions, accurately attribute credit.
  - They are considering the lab graph as a relatively safe shared space, different from DGF which tries to account for possibly adversarial actors.
  - Adding voting and various other forms of curation to help members in orientation over the graph.
  - Governance over the discourse graph protocol - adding new node types, deciding on credit attribution schemes, etc.
Broader open directions
- Different kinds of UX/graph tooling?
- How to find new DGs? How to keep updated regarding changes made to DGs? Thinking about using open social media protocols like ActivityPub.
- Federating DGs - how to query across multiple DGs?
- Exploring new types of edges and nodes.

- Matt shared with me this excellent presentation ([slides](https://roamresearch.com/#/app/akamatsulab/page/ud1vxU56v) + [video](https://share.descript.com/view/DcZY6qyNgKv)) he recently gave at the Allen Institute. - From what I can see, his lab is probably the most advanced in terms of actually using DGs for research - they haven’t actually written a paper directly using discourse graphs, but he runs the lab based on a shared collaborative DG, which forms the knowledge base underlying their research: (1) synthesizing existing knowledge (2) [they also want to extend this to ongoing research](https://roamresearch.com/#/app/akamatsulab/page/I1Y0Zb4o9) - Some interesting points that came up (well worth watching in full as well): - Matt mentioned a few times that [UX is key](https://roamresearch.com/#/app/akamatsulab/page/7ylN9m2dX), which jives with our intuitions as well. They also have a neat graphical UI to add nodes and connect them, a custom built add on to Roam that a developer created for them. - Not too surprisingly, it seems like there are some nice synergies between DGs and LLMs - they experimented with LLMs summarizing parts of the graph, writing results sections based on the graph, etc. Another interesting area could be searching the graph using natural language queries over LLMs (”claims we’ve made related to gene <X>”). - Matt mentions an [NSF grant on using DGs for molecular sciences knowledge synthesis](https://roamresearch.com/#/app/akamatsulab/page/pUiUtrQBa). - [Modular credit attribution](https://roamresearch.com/#/app/akamatsulab/page/wka3boHqu) is one of the benefits of using DGs. Matt also mentions that conveying research in the form of a shared DG (vs just research papers + code for example) could be made as a precondition to getting funding. - This bulletin board (around min 31:00) is a cool example of how the data can be aggregated both as a leaderboard and a way to help orient new researchers - seems related to some of the convos around topics at DGF, and how to organize/present forum data. ![image](/uploads/36b39df5a2dba3a219aee2230bb785a9/image.png) - In general, orientation over the graph is a challenge, the dashboards partially address it, but there is still a lot of work to be done on this front including using AI. - Some takeaways for DGF - This talk made me wonder again about what tooling/infrastructure to use for our experiment - They primarily use the Roam DG extension to create and navigate the graphs. - They use a Zotero - Roam integration to manage their references. Reference management seems like an important topic to consider. Maybe we could also use Zotero? Notion also works for this (and has a Zotero integration). - Re-raising the question regarding the forum custom graph UI - on one hand it will let us do new things that may be hard to add to Roam, but on the other hand it may take a while to get there, and usage friction will be high initially. Maybe we could try to use Roam or another similar tool initially and transition to our UI as it develops? - Some contributions / perspectives DGF can bring to the table - Social/reputation incentives are a direction they still haven’t looked into much yet, though I see this being discussed with Protocol Labs here: https://research.protocol.ai/blog/2023/discourse-graphs-and-the-future-of-science/. - Social incentives - even basic feedback in the form of likes over graph nodes, etc. - Reputation - ways to summarize member contributions, accurately attribute credit. - They are considering the lab graph as a relatively safe shared space, different from DGF which tries to account for possibly adversarial actors. - Adding voting and various other forms of curation to help members in orientation over the graph. - Governance over the discourse graph protocol - adding new node types, deciding on credit attribution schemes, etc. - Broader open directions - Different kinds of UX/graph tooling? - How to find new DGs? How to keep updated regarding changes made to DGs? Thinking about using open social media protocols like ActivityPub. - Federating DGs - how to query across multiple DGs? - Exploring new types of edges and nodes.

ronentk commented

2023-07-18 02:01:42 -05:00

(Migrated from gitlab.com)

assigned to @ronentk

ronentk commented

2023-07-18 02:02:45 -05:00

(Migrated from gitlab.com)

Next up - meeting Matt early August, feel free to share questions if you want me to ask him anything.