Are you applying for a Tribal Force member position or Tribal Force Leads?
I don’t mind doing TF Lead job in the part of tracking activity and approving new candidates, especially since for the first cohort I’m the only one with technical expertise for now. However, I cannot commit to a 40h/week job as a lead on top of my dev contribution. Therefore, for now I’m applying for the member position.
Describe your/the expert’s involvement in the Fei ecosystem and in the DeFi space more generally?
Since the early days of Fei I’ve been arguing here and in Discord discussions that multimillion decisions made in DeFi should be based on deeper understanding and prediction of underlying dynamics of protocols involved, both that of Fei and of its partners:
Implementing this vision I’ve been working lately on the tool for protocol exploration and comprehension.
It’s gonna be a tool for protocol analysis in the vein of what Gauntlet is doing, but based on a set of DeFi-native premises:
- Gauntlet is running their simulations with agent-based models representing users interacting with a protocol. A model of a user is based on a set of theoretical assumptions about user behaviour patterns. This approach was developed for TradFi, where the bulk of real life data is either not digitised at all (much of the b2c interaction happens offline) or isn’t available (much of market data is private). Hence, agent-based modelling with theoretical assumptions about incomplete data is justifiable. However, for DeFi agent-based modelling is a suboptimal legacy framework. Since all data about transactions and user interactions with the protocol is open and available for modelling, we can learn from real life data a model of a living protocol, or parts of it, and models of user interactions with it. Moreover, this model will be continuously fine-tuned with new data emerging.
- Transactions model is only half of the story. The other half is community sentiment manifested on twitter, discord and discourse. In the offline economy inflation expectations and consumer sentiment influence consumer behaviour and central banks of the world gauge it with polls, when modelling national economies. In DeFi we have the luxury to model community sentiment not with approximating polls, but again with real life data, while constantly fine-tuning the model.
We can learn from real life data models of onchain activity & that of community sentiment and then merge them to get a true-to-life DeFi-native model of a protocol, which can be constantly fine-tuned. Then it can be used to build tools for explorable + explainable DeFi: running stress tests and alternative scenarios, classifying protocols and tokens, detecting user behaviour patterns, making forecasts about certain protocol KPIs like TVL and that of partnering protocols.
I already made the first iteration of the model for gauging Twitter sentiment evolution towards some key DeFi protocols, including Fei. Under the hood is an autoencoding BERT-derivative language model trained on the corpus of tweets, then fine-tuned on a different corpus of tweets with sentiment labeled by human annotators. Then I scrapped Twitter feed with inclusion/exclusion @ # $ and some others for a particular protocol. Then I ran scrapped and preprocessed Twitter feed data through the language model and for each data point in the time series calculated rolling mean with a heuristic window (a sweet spot: less – too much noise, more – too few details):
I’ll be publishing more here: learn.klimchitsky.com
I’ve been doing this in my spare time with no funding. I’m also paying from my pocket for Google Cloud Services to run DL experiments, you can check out rates here: Preise | Vertex AI | Google Cloud
I’ve also been on the OA committee.
What skills/experience do you/they bring?
Deep learning for Natural Language Processing, deep learning on graphs: both community (members — vertices, communication — edges) and onchain (wallets — vertices, txns — edges) activities can be modelled as graphs.
Plus some industry connections to scale these efforts.
What are your expected contributions?
This work can go forward in a number of ways.
Trained models will be deployed to an endpoint such that apps like a (most basic) dashboard showing community sentiment real-time evolution or for certain dates/parameters can be built.
Advanced sentiment model
In the first iteration the most basic BERT_small-derivative (with some alterations) was used. A different architecture like ERNIE (demonstrates superior results in other natural language processing tasks) or an ensemble of architectures can replace underlying basic BERT-small model.
Resulting language model can be trained on a bigger corpus or a corpus of DeFi-specific vs general topic tweets.
E.g.: 175-billion-parameter GPT-3 was trained on a corpus of 570GB filtered texts from Common Crawl.
A classification task with 3 labels (positive, neutral, negative) is a good first step. A more advanced model can gauge community sentiment in a bigger/more complicated space of states or produce sentiment prediction in the form of coherent narrative. Custom data labelling could be required in this case.
In the first iteration Twitter feed is used as a source of data on community sentiment. Data from Discord, Discourse can augment the picture.
Enriched sentiment model (community social graph + sentiment)
Pure language models presume that all social interactions influence equally the state of community sentiment, which is clearly not the case. A message from a protocol founder vs average troll clearly should have different weights in the resulting picture.
One way to fix this is to use temporal graph network to represent an evolving community graph. It generates a temporal embedding (a real-valued vector) for each user (graph node) i. This embedding is a learnable function of their history of interactions and that of their n-hop neighbours (social ties akin to that measured by PageRank).
Thus a language model of community sentiment can be enriched with information about relative social weight of each community member.
Onchain txns model (transaction graph)
Temporal graph network can also be used to represent an evolving txns graph. Here, say for a given protocol, it generates a temporal embedding (a real-valued vector) for each wallet (graph node) i. This embedding is a learnable function of their history of transactions and that of their n-hop neighbours.
An ETH-wide representation graph can also be built.
Onchain txns model can be used to run predictions: running test scenarios for different assets/platforms/DAO decision parameter sets; classification: asset/protocol risk-benefit profiling; finding optimal monetary policy parameter set. Txns patterns forecasting. Fraudulent activity detection. Detecting other patterns/clusters providing certain insights and serving as a decision support tool for all stakeholders: DAOs, investors, community members, users. I invite everybody to brainstorm further possible applications.
Protocol supermodel
The most important and complicated part: a protocol supermodel integrating enriched sentiment and onchain txns models comes next.
Both models can be regarded as functions, each in their respective function spaces. Then temporal evolution of these models can be interpreted as operators on these function spaces and a protocol supermodel as morphisms between these operator spaces.
Also one of the key challenges here is to design a way to communicate this representation in a coherent and deep way such that a user could easily and productively explore and interact with the model. The appropriate UI/UX is in the pipeline.
Bonus question: What are some of the long-term goals with your contribution to the Fei ecosystem?
I think native tools for explorable + explainable DeFi as opposed to borrowed TradFi methods as done by Gauntlet or Aave are crucial for the future of Fei and DeFi at large.