Resilience and Wabi-Sabi

 
    
How to face life's challenges without letting stress and worry age you

What is resilience?

One thing that everyone with a clearly defined ikigai has in common is that they pursue their passion no matter what. They never give up, even when the cards seem stacked against them or they face one hurdle after another.
We're talking about resilience, a concept that has become influential among psychologists.
But resilience isn't just the ability to persevere. As we'll see in this chapter, it is also an outlook we can cultivate to stay focused on the important things in life rather than what is most urgent, and to keep ourselves from being carried away by negative emotions.
In the final section of the chapter, we'll explore techniques that go beyond resilience to cultivate antifragility.
Sooner or later, we all have to face difficult moments, and the way we do this can make a huge difference to our quality of life. Proper training for our mind, body, and emotional resilience is essential for confronting life's ups and downs.

七転び八起き
Nana korobi ya oki 

Fall seven times, rise eight.
—Japanese proverb

Resilience is our ability to deal with setbacks. The more resilient we are, the easier it will be to pick ourselves up and get back to what gives meaning to our lives.
Resilient people know how to stay focused on their objectives, on what matters, without giving in to discouragement. Their flexibility is the source of their strength: They know how to adapt to change and to reversals of fortune.
They concentrate on the things they can control and don't worry about those they can't.
In the words of the famous Serenity Prayer by Reinhold Niebuhr:

God, give us grace to accept with serenity the things that cannot be changed,
Courage to change the things which should be changed, and the Wisdom to distinguish the one from the other.

Emotional resilience through Buddhism and Stoicism

Siddhārtha Gautama (Buddha) was born a prince of Kapilavastu, Nepal, and grew up in a palace, surrounded by riches. At sixteen he married and had a child.
Not satisfied by his family's wealth, at twenty-nine he decided to try a different lifestyle and ran away from the palace to live as an ascetic. But it wasn't asceticism that he was looking for; it didn't offer the happiness and well-being he sought. Neither wealth nor extreme asceticism worked for him.
He realized that a wise person should not ignore life's pleasures. A wise person can live with these pleasures but should always remain conscious of how easy it is to be enslaved by them.
Zeno of Citium began his studies with the Cynics. The Cynics also led ascetic lives, leaving behind all earthly pleasures. They lived in the street, and the only thing they owned was the clothing on their backs.
Seeing that Cynicism did not give him a sense of well-being, Zeno abandoned its teachings to found the school of Stoicism, which centers on the idea that there is nothing wrong with enjoying life's pleasures as long as they do not take control of your life as you enjoy them. You have to be prepared for those pleasures to disappear. The goal is not to eliminate all feelings and pleasures from our lives, as in Cynicism, but to eliminate negative emotions.
Since their inception, one of the objectives of both Buddhism and Stoicism has been to control pleasure, emotions, and desires. Though the philosophies are very different, both aim to curb our ego and control our negative emotions.
Both Stoicism and Buddhism are, at their roots, methods for practicing well-being.

According to Stoicism, our pleasures and desires are not the problem. We can enjoy them as long as they don't take control of us. The Stoics viewed those who were able to control their emotions as virtuous.

What's the worst thing that could happen?

We finally land our dream job, but after a little while we are already hunting for a better one. We win the lottery and buy a nice car but then decide we can't live without a sailboat. We finally win the heart of the man or woman we've been pining for and suddenly find we have a wandering eye.
People can be insatiable.
The Stoics believed that these kinds of desires and ambitions are not worth pursuing. The objective of the virtuous person is to reach a state of tranquility (apatheia): the absence of negative feelings such as anxiety, fear, shame, vanity, and anger, and the presence of positive feelings such as happiness, love, serenity, and gratitude.
In order to keep their minds virtuous, the Stoics practiced something like negative visualization: They imagined the worst thing that could happen in order to be prepared if certain privileges and pleasures were taken from them.
To practice negative visualization, we have to reflect on negative events, but without worrying about them.
Seneca, one of the richest men in ancient Rome, lived a life of luxury but was, nonetheless, an active Stoic. He recommended practicing negative visualization every night before falling asleep. In fact, he not only imagined these negative situations, he actually put them into practice—for example, by living for a week without servants, or the food and drink he was used to as a wealthy man. As a result, he was able to answer the question “What's the worst thing that could happen?”

Meditating for healthier emotions

In addition to negative visualization and not giving in to negative emotions, another central tenet of Stoicism is knowing what we can control and what we can't, as we see in the Serenity Prayer. Worrying about things that are beyond our control accomplishes nothing.
We should have a clear sense of what we can change and what we can't, which in turn will allow us to resist giving in to negative emotions.
In the words of Epictetus, “It's not what happens to you, but how you react that matters.”

In Zen Buddhism, meditation is a way to become aware of our desires and emotions and thereby free ourselves from them. It is not simply a question of keeping the mind free of thoughts but instead involves observing our thoughts and emotions as they appear, without getting carried away by them. In this way, we train our minds not to get swept up in anger, jealousy, or resentment.
One of the most commonly used mantras in Buddhism focuses on controlling negative emotions: “Oṃ maṇi padme hūṃ ,” in which oṃ is the generosity that purifies the ego, ma is the ethics that purifies jealousy, ṇi is the patience that purifies passion and desire, pad is the precision that purifies bias, me is the surrender that purifies greed, and hūṃ is the wisdom that purifies hatred.

The here and now, and the impermanence of things

Another key to cultivating resilience is knowing in which time to live. Both Buddhism and Stoicism remind us that the present is all that exists, and it is the only thing we can control. Instead of worrying about the past or the future, we should appreciate things just as they are in the moment, in the now.
“The only moment in which you can be truly alive is the present moment,” observes the Buddhist monk Thich Nhat Hanh.
In addition to living in the here and now, the Stoics recommend reflecting on the impermanence of the things around us.
The Roman emperor Marcus Aurelius said that the things we love are like the leaves of a tree: They can fall at any moment with a gust of wind. He also said that changes in the world around us are not accidental but rather form part of the essence of the universe—a rather Buddhist notion, in fact.

We should never forget that everything we have and all the people we love will disappear at some point. This is something we should keep in mind, but without giving in to pessimism. Being aware of the impermanence of things does not have to make us sad; it should help us love the present moment and those who surround us.
“All things human are short-lived and perishable,” Seneca tells us.

The temporary, ephemeral, and impermanent nature of the world is central to every Buddhist discipline. Keeping this always in mind helps us avoid excessive pain in times of loss.

Wabi-sabi and ichi-go ichi-e

Wabi-sabi is a Japanese concept that shows us the beauty of the fleeting, changeable, and imperfect nature of the world around us. Instead of searching for beauty in perfection, we should look for it in things that are flawed, incomplete.
This is why the Japanese place such value, for example, on an irregular or cracked teacup. Only things that are imperfect, incomplete, and ephemeral can truly be beautiful, because only those things resemble the natural world.
A complementary Japanese concept is that of ichi-go ichi-e, which could be translated as “This moment exists only now and won't come again.” It is heard most often in social gatherings as a reminder that each encounter— whether with friends, family, or strangers—is unique and will never be repeated, meaning that we should enjoy the moment and not lose ourselves in worries about the past or the future.
The concept is commonly used in tea ceremonies, Zen meditation, and Japanese martial arts, all of which place emphasis on being present in the moment.
In the West, we've grown accustomed to the permanence of the stone buildings and cathedrals of Europe, which sometimes gives us the sense that nothing changes, making us forget about the passage of time. Greco-Roman architecture adores symmetry, sharp lines, imposing facades, and buildings and statues of the gods that outlast the centuries.
Japanese architecture, on the other hand, doesn't try to be imposing or perfect, because it is built in the spirit of wabi-sabi. The tradition of makingstructures out of wood presupposes their impermanence and the need for future generations to rebuild them. Japanese culture accepts the fleeting nature of the human being and everything we create.
The Grand Shrine of Ise,3 for example, has been rebuilt every twenty years for centuries. The most important thing is not to keep the building standing for generations, but to preserve customs and traditions—things that can withstand the passage of time better than structures made by human hands.
The key is to accept that there are certain things over which we have no control, like the passage of time and the ephemeral nature of the world around us.
Ichi-go ichi-e teaches us to focus on the present and enjoy each moment that life brings us. This is why it is so important to find and pursue our ikigai.
Wabi-sabi teaches us to appreciate the beauty of imperfection as an opportunity for growth.

Beyond resilience: Antifragility

As the legend goes, the first time Hercules faced the Hydra, he despaired when he discovered that cutting off one of its heads meant that two would grow back in its place. He would never be able to kill the beast if it got stronger with every wound.
As Nassim Nicholas Taleb explains in Antifragile: Things That Gain from Disorder,4 we use the word fragile to describe people, things, and organizations that are weakened when harmed, and the words robust and resilient for things that are able to withstand harm without weakening, but we don't have a word for things that get stronger when harmed (up to a point).
To refer to the kind of power possessed by the Hydra of Lerna, to talk about things that get stronger when they are harmed, Taleb proposes the term antifragile: “Antifragility is beyond resilience or robustness. The resilient resists shocks and stays the same; the antifragile gets better.”
Catastrophes and exceptional circumstances offer good models for explaining antifragility. In 2011 a tsunami hit the Tōhoku region of Japan, doing tremendous damage to dozens of cities and towns along the coast, most famously Fukushima.
When we visited the affected coast two years after the catastrophe, having driven for hours along cracked highways and past one empty gas station after another, we passed through several ghost towns whose streets had been taken over by the remnants of houses, piles of cars, and empty train stations. These towns were fragile spaces that had been forgotten by the government and could not recover on their own.
Other places, such as Ishinomaki and Kesennuma, suffered extensive damage but were rebuilt within a few years, thanks to the efforts of many.
Ishinomaki and Kesennuma showed how resilient they were in their ability to return to normal after the catastrophe.
The earthquake that caused the tsunami also affected the Fukushima nuclear power plant. The Tokyo Electric Power Company engineers working at the plant were not prepared to recover from that kind of damage. The Fukushima nuclear facility is still in a state of emergency and will be for decades to come. It demonstrated its fragility in the face of an unprecedented catastrophe.
The Japanese financial markets closed minutes after the earthquake.
Which businesses did the best in the aftermath? Stock in big construction companies has been steadily on the rise since 2011; the need to rebuild the entire coast of Tōhoku is a boon for construction. In this case, Japanese construction companies are antifragile, since they benefited enormously from the catastrophe.
Now let's take a look at how we can apply this concept to our daily lives.
How can we be more antifragile?

Step 1: Create redundancies

Instead of having a single salary, try to find a way to make money from your hobbies, at other jobs, or by starting your own business. If you have only one salary, you might be left with nothing should your employer run into trouble, leaving you in a position of fragility. On the other hand, if you have several options and you lose your primary job, it might just happen that you end up dedicating more time to your secondary job, and maybe even make moremoney at it. You would have beaten that stroke of bad luck and would be, in that case, antifragile.
One hundred percent of the seniors we interviewed in Ogimi had a primary and a secondary occupation. Most of them kept a vegetable garden as a secondary job, and sold their produce at the local market.
The same idea goes for friendships and personal interests. It's just a matter, as the saying goes, of not putting all your eggs in one basket.
In the sphere of romantic relationships, there are those who focus all their energy on their partner and make him or her their whole world. Those people lose everything if the relationship doesn't work out, whereas if they've cultivated strong friendships and a full life along the way, they'll be in a better position to move on at the end of a relationship. They'll be antifragile.
Right now you might be thinking, “I don't need more than one salary, and I'm happy with the friends I've always had. Why should I add anything new?”
It might seem like a waste of time to add variation to our lives, because extraordinary things don't ordinarily happen. We slip into a comfort zone. But the unexpected always happens, sooner or later.

Step 2: Bet conservatively in certain areas and take many small risks in others

The world of finance turns out to be very useful in explaining this concept. If you have $10,000 saved up, you might put $9,000 of that into an index fund or fixed-term deposit, and invest the remaining $1,000 in ten start-ups with huge growth potential—say, $100 in each.
One possible scenario is that three of the companies fail (you lose $300), the value of three other companies goes down (you lose another $100 or $200), the value of three goes up (you make $100 or $200), and the value of one of the start-ups increases twenty-fold (you make nearly $2,000, or maybe even more).
You still make money, even if three of the businesses go completely belly- up. You've benefited from the damage, just like the Hydra.
The key to becoming antifragile is taking on small risks that might lead to great reward, without exposing ourselves to dangers that might sink us, such as investing $10,000 in a fund of questionable reputation that we saw advertised in the newspaper.

Step 3: Get rid of the things that make you fragile

We're taking the negative route for this exercise. Ask yourself: What makes me fragile? Certain people, things, and habits generate losses for us and make us vulnerable. Who and what are they? When we make our New Year's resolutions, we tend to emphasize adding new challenges to our lives. It's great to have this kind of objective, but setting “good riddance” goals can have an even bigger impact. For example:

# Stop snacking between meals

# Eat sweets only once a week

# Gradually pay off all debt

# Avoid spending time with toxic people

# Avoid spending time doing things we don't enjoy, simply because we feel obligated to do them

# Spend no more than twenty minutes on Facebook per day

To build resilience into our lives, we shouldn't fear adversity, because each setback is an opportunity for growth. If we adopt an antifragile attitude, we'll find a way to get stronger with every blow, refining our lifestyle and staying focused on our ikigai.
Taking a hit or two can be viewed as either a misfortune or an experience that we can apply to all areas of our lives, as we continually make corrections and set new and better goals. As Taleb writes in Antifragile, “We need randomness, mess, adventures, uncertainty, self-discovery, hear traumatic episodes, all these things that make life worth living.” We encourage those interested in the concept of antifragility to read Nassim Nicholas Taleb's Antifragile.
Life is pure imperfection, as the philosophy of wabi-sabi teaches us, and the passage of time shows us that everything is fleeting, but if you have a clear sense of your ikigai, each moment will hold so many possibilities that it will seem almost like an eternity.

Source: Chapter 9 from the book "Ikigai" by Hector Garcia

Tags: Book Summary,Emotional Intelligence,Buddhism,

React Native Books (Jul 2024)

To See All Tech Related Book Lists: Index of Book Lists And Downloads
Download Books

1.
Learning React Native: Building Native Mobile Apps with JavaScript
Bonnie Eisenman, 2015

2.
React Native in Action: Developing IOS and Android Apps with JavaScript
Nader Dabit, 2019

3.
React and React Native
Adam Boduch, 2017

4.
Fullstack React Native: Create Beautiful Mobile Apps with JavaScript and React Native
Anthony Accomazzo, 2019

5.
React Native for Mobile Development: Harness the Power of React Native to Create Stunning IOS and Android Applications
Abhishek Nalwaya, 2019

6.
React and React Native: A Complete Hands-on Guide to Modern Web and Mobile Development with React.js
Adam Boduch, 2020

7.
React Native Cookbook: Recipes for Solving Common React Native Development Problems
Dan Ward, 2019

8.
Mastering React Native: A Beginner's Guide
2022

9.
Professional React Native: Expert Techniques and Solutions for Building High-quality, Cross-platform, Production-ready Apps
Alexander Benedikt Kuttig, 2022

10.
React and React Native: Build Cross-platform JavaScript Applications with Native Power for the Web, Desktop, and Mobile
Adam Boduch, 2022

11.
Simplifying State Management in React Native: Master State Management from Hooks and Context Through to Redux, MobX, XState, Jotai and React Query
Aleksandra Desmurs-Linczewska, 2023

12.
JavaScript Everywhere: Building Cross-Platform Applications with GraphQL, React, React Native, and Electron
Adam D. Scott, 2020

13.
Practical React Native: Build Two Full Projects and One Full Game Using React Native
Frank Zammetti, 2018

14.
Mastering React Native
Cybellium Ltd

15.
React Native for IOS Development
Abhishek Nalwaya, 2015

16.
Learning React: Modern Patterns for Developing React Apps
Alex Banks, 2020

17.
React: Cross-Platform Application Development with React Native: Build 4 Real-world Apps with React Native
Emilio Rodriguez Martinez, 2018

18.
React Native By Example
Richard Kho, 2017

19.
React Native Cookbook: Bringing the Web to Native Platforms
Jonathan Lebensold, 2018

20.
React Key Concepts: Consolidate Your Knowledge of React's Core Features
Maximilian Schwarzmuller, 2022

21.
Hands-On Design Patterns with React Native: Proven Techniques and Patterns for Efficient Native Mobile Development with JavaScript
Mateusz Grzesiukiewicz, 2018

22.
React in Action
Mark Tielens Thomas, 2018

23.
React: Up & Running: Building Web Applications
Stoyan Stefanov, 2016

24.
Beginning React with Hooks
Greg Lim, 2020

25.
React Design Patterns and Best Practices
Michele Bertoli, 2017

26.
Mastering React Native
Eric Masiello, 2017

27.
React Native Cookbook
Stan Bershadskiy, 2016

28.
Beginning React (incl. Redux and React Hooks)
Greg Lim, 2017

29.
The Road to Learn React: Your Journey to Master Plain Yet Pragmatic React. Js
Robin Wieruch, 2017

30.
React Native - Building Mobile Apps with JavaScript
Vladimir Novick, 2017

31.
React Native Blueprints: Create Eight Exciting Native Cross-platform Mobile Applications with JavaScript
Emilio Rodriguez Martinez, 2017

32.
Learn All about React Native
Innoware Pjp, 2023

33.
Creating Apps with React Native: Deliver Cross-Platform 0 Crash, 5 Star Apps
M. Holmes He, 2022

34.
React Native Tutorial: How to Start with React Native. Beginners Guide Book
Nicholas Brown, 2016

35.
React: Quickstart Step-By-Step Guide to Learning React Javascript Library (React. Js, Reactjs, Learning React JS, React Javascript, React Programming)
Lionel Lopez, 2017

36.
React Quickly: Painless Web Apps with React, JSX, Redux, and GraphQL
Azat Mardan, 2017

37.
React Cookbook: Create Dynamic Web Apps with React Using Redux, Webpack, Node.js, and GraphQL
Carlos Santana Roldán, 2018

38.
React Design Patterns and Best Practices: Design, Build and Deploy Production-ready Web Applications Using Standard Industry Practices, 2nd Edition
Carlos Santana Roldán, 2019

39.
Beginning ReactJS Foundations Building User Interfaces with ReactJS: An Approachable Guide
Chris Minnick, 2022

40.
Lightning-Fast Mobile App Development with Galio: Build Stylish Cross-platform Mobile Apps with Galio and React Native
Alin Gheorghe, 2021

41.
The Road to React: With React 18 and React Hooks : Required Knowledge: JavaScript
Robin Wieruch, 2017

42.
Getting Started with React Native
Tom Bray, 2015

43.
Learning React: Functional Web Development with React and Redux
Alex Banks, 2017

44.
Fullstack React: The Complete Guide to ReactJS and Friends
Nate Murray, 2017

45.
Learning React: A Hands-On Guide to Building Web Applications Using React and Redux
Kirupa Chinnathambi, 2017

46.
Pro React 16
Adam Freeman, 2019

47.
Getting Started with React
Danillo Corvalan, 2016

48.
React Development Using TypeScript: Modern Web App Development Using Advanced React Techniques (English Edition)
Pranali Dahale, 2024

49.
React Projects: Build Advanced Cross-platform Projects with React and React Native to Become a Professional Developer
Roy Derks, 2022

50.
React Native for Mobile Development (2 in 1 eBooks)
2023

Tags: Technology,List of Books,JavaScript,React Native,

Interview Preparation For Wipro - Questions on Large Language Models and Generative AI (Jul 2024)

To See All Interview Preparation Articles: Index For Interviews Preparation

Prompt

I am a data scientist applying for a job in Wipro.
The job description is given below in triple quotes:

'''% Contribute to the design and implementation of state-of-the-art AI solutions.

% Assist in the development and implementation of AI models and systems, leveraging techniques such as Language Models (LLMs) and generative AI.
Collaborate with stakeholders to identify business opportunities and define AI project goals.

% Stay updated with the latest advancements in generative AI techniques, such as LLMs, and evaluate their potential applications in solving enterprise challenges.

% Utilize generative AI techniques, such as LLMs, to develop innovative solutions for enterprise industry use cases.

% Integrate with relevant APIs and libraries, such as Azure Open AI GPT models and Hugging Face Transformers, to leverage pre-trained models and enhance generative AI capabilities.

% Implement and optimize end-to-end pipelines for generative AI projects, ensuring seamless data processing and model deployment.

% Utilize vector databases, such as Redis, and NoSQL databases to efficiently handle large-scale generative AI datasets and outputs.

% Implement similarity search algorithms and techniques to enable efficient and accurate retrieval of relevant information from generative AI outputs.

% Collaborate with domain experts, stakeholders, and clients to understand specific business requirements and tailor generative AI solutions accordingly.

% Conduct research and evaluation of advanced AI techniques, including transfer learning, domain adaptation, and model compression, to enhance performance and efficiency.

% Establish evaluation metrics and methodologies to assess the quality, coherence, and relevance of generative AI outputs for enterprise industry use cases.

% Ensure compliance with data privacy, security, and ethical considerations in AI applications.

% Leverage data engineering skills to curate, clean, and preprocess large-scale datasets for generative AI applications.'''

Please let me know what to expect in this interview, what to expect from interviewers. What kind of questions might be asked?

Response

For an interview at Wipro for a data scientist position focusing on AI solutions and generative AI, you can expect questions that assess both your technical skills and your ability to apply these skills to business problems. Here are some areas and example questions to help you prepare:

Technical Skills

AI Models and Techniques:

1. Explain the working principles of Language Models (LLMs) and generative AI.

2. How many types of attention mechanisms are there?

3. How do you fine-tune a pre-trained language model for a specific task?

4. Describe your experience with transfer learning and domain adaptation.

Generative AI Solutions:

5. How have you used generative AI to solve real-world problems? Can you provide a specific example?

6. What are some challenges you've faced when implementing generative AI models, and how did you overcome them?

APIs and Libraries:

7. Describe your experience with APIs and libraries like Azure Open AI GPT models and Hugging Face Transformers.

8. How do you integrate pre-trained models into your AI pipelines?

9. How to fine-tune the Azure OpenAI's GPT-4 model?

Data Engineering:

10. How do you preprocess large-scale datasets for AI applications?

11. Explain your experience with vector databases like Redis and NoSQL databases.

Model Deployment:

12. What steps do you take to ensure seamless data processing and model deployment in AI projects?

13. How do you optimize end-to-end AI pipelines?

Evaluation and Metrics:

14. What evaluation metrics do you use to assess the quality and relevance of generative AI outputs?

15. How do you conduct research to stay updated with the latest advancements in AI?

Business and Stakeholder Collaboration

Business Problem Identification:

How do you collaborate with stakeholders to identify business opportunities for AI projects?
Can you describe a situation where you defined AI project goals based on business needs?

Solution Tailoring:

How do you tailor generative AI solutions to meet specific business requirements?
Give an example of how you collaborated with domain experts to develop an AI solution.

Ethical Considerations:

How do you ensure compliance with data privacy and security in your AI applications?
What ethical considerations do you keep in mind when developing AI solutions?

Research and Development

Advanced AI Techniques:

How do you leverage techniques like model compression and similarity search algorithms in your projects?
Describe a project where you used advanced AI techniques to enhance model performance.

Example Questions

Technical Deep-Dive:

Can you walk us through a recent AI project you worked on, focusing on the techniques you used and the outcomes?
How do you handle large-scale datasets in your AI projects?

Practical Application:

How would you use generative AI to improve a business process at Wipro?
What are the potential applications of LLMs in enterprise industry use cases?

Problem-Solving:

Describe a challenging AI project you worked on. What was the challenge, and how did you address it?
How do you ensure the AI models you develop are robust and reliable?
Preparing for these types of questions and thinking through relevant examples from your experience will help you demonstrate your expertise and fit for the role.

~~~ ~~~ ~~~ ~~~ ~~~

Solved Questions

Prompt

Explain the working principles of Language Models (LLMs) and generative AI.

Response

Working Principles of Language Models (LLMs)

Language Models (LLMs) are a subset of artificial intelligence models designed to understand and generate human language. They are primarily based on deep learning architectures, such as Transformers. Here are the key working principles:

Training on Large Datasets:

LLMs are trained on vast corpora of text data. This data can include books, articles, websites, and other text sources. The large amount of data helps the model learn the intricacies of language, such as grammar, context, semantics, and common usage patterns.


Tokenization:

Before training, the text data is broken down into smaller units called tokens. Tokens can be words, subwords, or characters, depending on the tokenization strategy. This process converts the text into a format that the model can process.


Transformer Architecture:

LLMs, like GPT-3 and GPT-4, use the Transformer architecture. Transformers consist of layers of self-attention mechanisms and feedforward neural networks. The self-attention mechanism allows the model to weigh the importance of different words in a sentence when making predictions, capturing long-range dependencies and contextual information.


Self-Attention Mechanism:

Self-attention allows the model to focus on different parts of the input sequence when processing each token. It calculates attention scores that determine how much weight to give to each token in the sequence, enabling the model to capture contextual relationships.


Pre-training and Fine-tuning:

Pre-training involves training the model on a large text corpus using unsupervised learning techniques, such as predicting the next word in a sentence (language modeling) or filling in masked words (masked language modeling).

Fine-tuning involves training the pre-trained model on a smaller, task-specific dataset using supervised learning. This step adapts the model to specific tasks, such as text classification, question answering, or text generation.

Working Principles of Generative AI

Generative AI focuses on creating new content, such as text, images, or audio, that is similar to the content it was trained on. Here are the key working principles:

Generative Models:

Generative models are designed to generate new data samples from the learned distribution of the training data. Examples include Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and LLMs.


Probabilistic Modeling:

Generative AI models learn the underlying probability distribution of the training data. This allows them to generate new data points that are statistically similar to the training data. For text, this means generating coherent and contextually appropriate sentences.


Latent Space Representation:

Many generative models, like VAEs, learn a compressed latent space representation of the data. This latent space captures the essential features of the data, and the model can sample from this space to generate new instances.


Training Techniques:

Generative models are trained using various techniques. GANs, for example, involve a generator that creates fake data and a discriminator that tries to distinguish between real and fake data. The generator improves by trying to fool the discriminator, leading to better and more realistic data generation.


Applications in Text Generation:

LLMs can generate human-like text by predicting the next word in a sequence based on the context provided by the preceding words. This ability allows them to create coherent and contextually relevant sentences, paragraphs, or even entire articles.
Fine-tuning for Specific Tasks:

Similar to LLMs, generative AI models can be fine-tuned for specific tasks. For example, an LLM can be fine-tuned to generate creative writing, code, or even technical documentation by training it on domain-specific data.

Integration and Optimization

APIs and Libraries:

Generative AI solutions often integrate with APIs and libraries like Hugging Face Transformers and Azure Open AI to leverage pre-trained models and enhance capabilities. These integrations allow for more efficient and scalable implementations.


End-to-End Pipelines:

Implementing end-to-end pipelines involves data preprocessing, model training, inference, and deployment. Ensuring seamless integration and optimization of these pipelines is crucial for the practical application of generative AI.


Evaluation Metrics:

Evaluating generative AI outputs involves metrics such as coherence, relevance, diversity, and quality. These metrics help in assessing the performance and suitability of the generated content for specific use cases.
By understanding these principles, you can explain the fundamental workings of LLMs and generative AI, showcasing your expertise in these areas during your interview.



Prompt

How many types of attention mechanisms are there?

Response

Attention mechanisms are crucial in many modern deep learning architectures, particularly in natural language processing (NLP) and computer vision. Here are several types of attention mechanisms commonly used:

Self-Attention:

Description: Each element in a sequence attends to all other elements, including itself.
Example: Transformer models use self-attention in both encoder and decoder layers.
Application: NLP tasks like machine translation, text summarization.


Soft Attention:

Description: Assigns a probability distribution over the input sequence and calculates a weighted sum of input elements.
Example: Used in the Bahdanau Attention mechanism for neural machine translation.
Application: Image captioning, machine translation.


Hard Attention:

Description: Selects one element from the input sequence based on the highest probability (stochastic process).
Example: Used less frequently due to the non-differentiable nature, often requires reinforcement learning techniques.
Application: Situations where discrete decisions are necessary.


Global Attention:

Description: Considers all input elements when computing the attention weights.
Example: Global Attention mechanism in Seq2Seq models.
Application: Text generation, language modeling.


Local Attention:

Description: Focuses on a subset of input elements, reducing computational complexity.
Example: Local Attention in Seq2Seq models.
Application: Speech recognition, where only a local context is relevant.


Multi-Head Attention:

Description: Uses multiple attention mechanisms in parallel, each with different parameters, and concatenates their outputs.
Example: Transformer models extensively use multi-head attention.
Application: All Transformer-based models, including BERT, GPT.


Hierarchical Attention:

Description: Applies attention at multiple levels of the input, such as word-level and sentence-level attention.
Example: Hierarchical Attention Networks (HANs) for document classification.
Application: Document classification, sentiment analysis.


Cross-Attention:

Description: One sequence (e.g., a query) attends to another sequence (e.g., key-value pairs).
Example: Decoder attending to encoder outputs in Transformer models.
Application: Machine translation, sequence-to-sequence tasks.


Scaled Dot-Product Attention:

Description: Computes the dot product between query and key vectors, scales by the square root of the dimension, and applies a softmax function.
Example: Core component of the Transformer model's attention mechanism.
Application: All Transformer-based models, including BERT, GPT.


Additive Attention:

Description: Computes attention scores using a feedforward network with a single hidden layer.
Example: Bahdanau Attention in Seq2Seq models.
Application: Machine translation, text summarization.


Memory-Augmented Attention:

Description: Uses external memory to store intermediate representations, allowing the model to attend over a potentially unlimited memory.
Example: Neural Turing Machines, Differentiable Neural Computers.
Application: Tasks requiring long-term dependencies, such as algorithmic tasks and reasoning.


Each of these attention mechanisms has its strengths and is chosen based on the specific requirements of the task at hand. Understanding these mechanisms can help you better design and implement models for various AI applications.


Prompt

How do you fine-tune a pre-trained language model for a specific task?

Response

Fine-tuning a pre-trained language model for a specific task involves several steps. Here's a comprehensive guide to the process:

1. Select a Pre-trained Model:
Choose an appropriate pre-trained model based on your task. Popular choices include BERT, GPT-3, GPT-4, RoBERTa, and T5. These models are available in libraries like Hugging Face Transformers.

2. Prepare Your Data:

Data Collection: Gather a dataset relevant to your specific task. For instance, if you are fine-tuning for text classification, you need labeled text data.

Data Preprocessing: Clean and preprocess the data. This may include tokenization, removing special characters, and lowercasing text. Use the tokenizer associated with the chosen pre-trained model.

3. Setup the Environment:

Install necessary libraries and frameworks. For example, using Hugging Face Transformers:

!pip install transformers datasets

4. Load the Pre-trained Model and Tokenizer:


    from transformers import AutoModelForSequenceClassification, AutoTokenizer

    model_name = 'bert-base-uncased'  # Example model
    model = AutoModelForSequenceClassification.from_pretrained(model_name)
    tokenizer = AutoTokenizer.from_pretrained(model_name)

5. Tokenize the Data:
Tokenize the input data using the tokenizer.


    def tokenize_function(examples):
        return tokenizer(examples['text'], padding="max_length", truncation=True)

    tokenized_datasets = dataset.map(tokenize_function, batched=True)

6. Prepare Data Loaders:
Split the dataset into training and validation sets and prepare data loaders.


    from torch.utils.data import DataLoader

    train_dataset = tokenized_datasets['train']
    eval_dataset = tokenized_datasets['validation']

    train_dataloader = DataLoader(train_dataset, shuffle=True, batch_size=16)
    eval_dataloader = DataLoader(eval_dataset, batch_size=16)

7. Define the Training Arguments:

Specify the training arguments, including learning rate, number of epochs, and evaluation strategy.


    from transformers import TrainingArguments

    training_args = TrainingArguments(
        output_dir='./results',          
        evaluation_strategy="epoch",
        learning_rate=2e-5,
        per_device_train_batch_size=16,
        per_device_eval_batch_size=16,
        num_train_epochs=3,
        weight_decay=0.01,
    )

8. Create a Trainer:
Use the Trainer class to manage the training process.

python
Copy code
from transformers import Trainer

trainer = Trainer(
    model=model,                         
    args=training_args,                  
    train_dataset=train_dataset,         
    eval_dataset=eval_dataset,           
)
9. Train the Model:
Start the training process.

python
Copy code
trainer.train()
10. Evaluate the Model:
Evaluate the model on the validation set to monitor its performance.

python
Copy code
eval_results = trainer.evaluate()
print(f"Evaluation results: {eval_results}")
11. Save the Fine-Tuned Model:
Save the fine-tuned model for future use.

python
Copy code
model.save_pretrained('./fine-tuned-model')
tokenizer.save_pretrained('./fine-tuned-model')
Example: Fine-Tuning BERT for Text Classification

~~~ ~~~ ~~~ ~~~ ~~~

Here is a more detailed example of fine-tuning BERT for a text classification task using the Hugging Face library:

Install Necessary Libraries:

pip install transformers datasets


Load Dataset:


from datasets import load_dataset

dataset = load_dataset('imdb')


Load Pre-trained Model and Tokenizer:


from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_name = 'bert-base-uncased'
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)
tokenizer = AutoTokenizer.from_pretrained(model_name)


Tokenize Data:


def tokenize_function(examples):
    return tokenizer(examples['text'], padding="max_length", truncation=True)

tokenized_datasets = dataset.map(tokenize_function, batched=True)


Prepare Data Loaders:


from torch.utils.data import DataLoader

train_dataset = tokenized_datasets['train']
eval_dataset = tokenized_datasets['test']

train_dataloader = DataLoader(train_dataset, shuffle=True, batch_size=16)
eval_dataloader = DataLoader(eval_dataset, batch_size=16)



Define Training Arguments:


from transformers import TrainingArguments

training_args = TrainingArguments(
    output_dir='./results',          
    evaluation_strategy="epoch",
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    num_train_epochs=3,
    weight_decay=0.01,
)


Create a Trainer:


from transformers import Trainer

trainer = Trainer(
    model=model,                         
    args=training_args,                  
    train_dataset=train_dataset,         
    eval_dataset=eval_dataset,           
)


Train the Model:

trainer.train()


Evaluate the Model:


eval_results = trainer.evaluate()
print(f"Evaluation results: {eval_results}")

Save the Fine-Tuned Model:


model.save_pretrained('./fine-tuned-model')
tokenizer.save_pretrained('./fine-tuned-model')

This step-by-step guide should help you fine-tune a pre-trained language model for a specific task, ensuring it performs well on your domain-specific data.



Prompt

Describe your experience with transfer learning and domain adaptation.

Response

Transfer Learning
Transfer learning involves leveraging knowledge from a pre-trained model on a large dataset to solve a related task on a smaller dataset. This approach is particularly useful when there is limited labeled data for the specific task at hand. Here’s a breakdown of my experience:

Pre-trained Models:

I have extensively used pre-trained models like BERT, GPT-3, and GPT-4 for various NLP tasks. These models are trained on large datasets and capture a wide range of language patterns and representations.

Fine-tuning:

I have fine-tuned pre-trained models on domain-specific datasets to adapt them to specific tasks such as text classification, sentiment analysis, and named entity recognition. This involves additional training on a smaller, labeled dataset specific to the task while preserving the learned features from the original pre-training.

Applications:

In a recent project, I fine-tuned BERT for a sentiment analysis task on a customer review dataset. This significantly improved the model’s performance compared to training a model from scratch, due to BERT's ability to understand context and semantics.
For a text generation task, I fine-tuned GPT-3 to generate product descriptions based on specific attributes. This involved providing domain-specific training data to the model, which enabled it to produce coherent and contextually relevant descriptions.

Domain Adaptation

Domain adaptation is a type of transfer learning where a model trained in one domain (source domain) is adapted to work well in another domain (target domain). This is particularly challenging when there are differences between the source and target domains. Here’s a summary of my experience:

Data Distribution Differences:

I have worked on projects where the source and target domains had different data distributions. For instance, adapting a model trained on news articles to work on scientific papers. This required addressing issues such as vocabulary differences and varying styles of writing.

Techniques Used:

Fine-tuning with Target Data: I fine-tuned models with a small amount of labeled data from the target domain. For example, adapting a general text classification model to classify medical texts by fine-tuning it on a labeled dataset of medical articles.
Feature Alignment: I employed techniques like domain adversarial training to align features between source and target domains. This helps the model learn domain-invariant features that are useful for the target domain.
Unsupervised Domain Adaptation: In cases where labeled data from the target domain was scarce or unavailable, I used unsupervised domain adaptation techniques such as using pseudo-labeling and self-training to iteratively improve the model's performance on the target domain.

Real-World Example:

In a recent project, I adapted a customer feedback analysis model trained on general product reviews to work on reviews for a specific product category (e.g., electronics). This involved fine-tuning the model on a smaller dataset of electronics reviews and using domain adaptation techniques to handle vocabulary and context differences.
Key Outcomes and Benefits
Improved Performance: Transfer learning and domain adaptation have consistently led to improved performance on specific tasks compared to training models from scratch. Leveraging pre-trained models reduces the amount of labeled data required and shortens the training time.
Reduced Costs: These techniques help in reducing computational costs and time required for model training by reusing existing models and adapting them to new tasks or domains.
Versatility: The ability to quickly adapt models to new domains makes them versatile and applicable to a wide range of problems, from NLP to computer vision and beyond.

Overall, my experience with transfer learning and domain adaptation has enabled me to efficiently tackle various AI challenges, delivering robust and high-performing models across different applications and domains.

Prompt

Describe transfer learning and domain adaptation with examples and use cases.

Response

Transfer Learning
Transfer learning is a machine learning technique where a model developed for a specific task is reused as the starting point for a model on a second task. It leverages knowledge gained from a large, diverse dataset to improve the performance of models on smaller, domain-specific datasets.

Examples and Use Cases

Image Classification:

Example: Using a pre-trained model like VGG16 or ResNet, which has been trained on ImageNet (a large dataset with millions of labeled images across thousands of categories), and fine-tuning it for a specific task, such as classifying medical images.
Use Case: A healthcare startup uses a pre-trained ResNet model to detect skin cancer from dermatoscopic images. By fine-tuning the pre-trained model on a smaller dataset of labeled medical images, the startup can achieve high accuracy with less training data and computational resources.
Natural Language Processing (NLP):

Example: Using BERT (Bidirectional Encoder Representations from Transformers), which has been pre-trained on a large corpus of text, and fine-tuning it for a specific task like sentiment analysis or named entity recognition.
Use Case: A customer service platform fine-tunes BERT on a dataset of customer reviews to develop a sentiment analysis model. This model can accurately determine the sentiment of new reviews, helping the platform identify areas of improvement based on customer feedback.
Speech Recognition:

Example: Utilizing a pre-trained speech recognition model like DeepSpeech, which has been trained on a diverse dataset of spoken language, and fine-tuning it for a specific language or accent.
Use Case: An educational technology company adapts DeepSpeech to recognize accented English commonly spoken by non-native speakers. This enables more accurate transcription and enhances the learning experience for users worldwide.
Domain Adaptation
Domain adaptation is a subfield of transfer learning where a model trained in one domain (source domain) is adapted to work well in another domain (target domain). This is particularly useful when there are differences in data distribution between the source and target domains.

Examples and Use Cases
Adapting Text Classification Models:

Example: A text classification model trained on news articles (source domain) is adapted to classify scientific papers (target domain). The vocabulary and writing style in scientific papers differ significantly from news articles.
Use Case: A research organization adapts a news article classifier to identify relevant scientific papers for literature reviews. By fine-tuning the model on a smaller dataset of labeled scientific papers, the organization improves the classifier's accuracy in the target domain.
Cross-Domain Sentiment Analysis:

Example: A sentiment analysis model trained on movie reviews (source domain) is adapted to analyze product reviews (target domain). The expressions and contextual cues in product reviews differ from those in movie reviews.
Use Case: An e-commerce platform adapts a movie review sentiment analysis model to analyze customer feedback on products. This helps the platform quickly identify and address customer satisfaction issues.
Adapting Visual Models:

Example: A computer vision model trained on everyday objects (source domain) is adapted to recognize industrial machinery (target domain). The visual characteristics of industrial machinery are different from those of everyday objects.
Use Case: A manufacturing company adapts a general object detection model to monitor and detect defects in industrial machinery. This helps the company maintain equipment and prevent costly downtimes.

Techniques for Domain Adaptation

Fine-Tuning with Target Data:

Description: Fine-tuning the pre-trained model with a small amount of labeled data from the target domain.
Example: Fine-tuning a text classification model with a small labeled dataset of scientific papers.


Feature Alignment:

Description: Aligning the feature distributions of the source and target domains using techniques like domain adversarial training.
Example: Using domain adversarial neural networks (DANNs) to learn domain-invariant features for cross-domain sentiment analysis.


Unsupervised Domain Adaptation:

Description: Adapting the model without labeled target domain data by leveraging techniques like pseudo-labeling and self-training.
Example: Iteratively improving a sentiment analysis model using unlabeled product reviews by generating pseudo-labels and refining the model.
Key Outcomes and Benefits
Improved Performance: Leveraging pre-trained models or adapting models to new domains generally results in better performance compared to training models from scratch.
Reduced Training Time: Transfer learning and domain adaptation significantly reduce the amount of data and time required for training, enabling quicker deployment of models.
Resource Efficiency: These techniques help in efficiently utilizing computational resources and labeled data, making AI solutions more accessible and scalable.
By understanding and applying transfer learning and domain adaptation, organizations can build robust, high-performing models tailored to specific tasks and domains, thereby achieving better results with fewer resources.

Prompt

What are some challenges you've faced when implementing generative AI models, and how did you overcome them?

Response

Implementing generative AI models presents several challenges, ranging from technical issues to ethical considerations. Here are some of the challenges I've faced and the strategies I've used to overcome them:

1. Data Quality and Quantity
Challenge:

Generative AI models, such as GPT-3 and GPT-4, require large amounts of high-quality data to produce meaningful and coherent outputs. Insufficient or poor-quality data can lead to models that generate irrelevant or nonsensical content.
Solution:

Data Augmentation: I used data augmentation techniques to artificially increase the size of the training dataset. This included paraphrasing, synonym replacement, and back-translation for text data.
Data Cleaning: I implemented rigorous data cleaning processes to remove noise, correct errors, and ensure consistency. This involved automated scripts for preprocessing and manual review for critical datasets.
Synthetic Data Generation: For specific tasks, I generated synthetic data that closely mimicked the target domain. This was particularly useful in cases where real data was scarce.

2. Computational Resources
Challenge:

Training large generative models is computationally expensive and requires significant resources, including GPUs or TPUs, which can be costly and time-consuming.
Solution:

Cloud Computing: I leveraged cloud-based solutions like AWS, Google Cloud, and Azure to access powerful computational resources on-demand, optimizing cost and scalability.
Model Optimization: I employed techniques like model pruning, quantization, and distillation to reduce the model size and computational requirements without significantly compromising performance.
Distributed Training: I used distributed training frameworks (e.g., Horovod, PyTorch Distributed) to parallelize the training process across multiple GPUs or nodes, reducing the overall training time.

3. Model Fine-Tuning and Adaptation
Challenge:

Fine-tuning pre-trained generative models for specific tasks or domains can be tricky, as it requires careful balancing to avoid overfitting and to ensure the model adapts well to the new data.
Solution:

Layer-wise Learning Rate Adjustment: I used different learning rates for different layers of the model, allowing the lower layers to retain general knowledge while adapting the higher layers to the specific task.
Regularization Techniques: I applied regularization techniques like dropout, weight decay, and early stopping to prevent overfitting and ensure the model generalizes well to unseen data.
Evaluation and Iteration: I continuously evaluated the model's performance on a validation set and iteratively refined the fine-tuning process based on the results, adjusting hyperparameters and data preprocessing steps as needed.

4. Generating Coherent and Relevant Outputs
Challenge:

Ensuring that the generative model produces coherent, contextually relevant, and meaningful outputs is challenging, especially for tasks requiring high accuracy and relevance.
Solution:

Prompt Engineering: I designed and refined prompts to guide the model's generation process more effectively. This included providing clear instructions, context, and examples within the prompts.
Controlled Generation: I implemented controlled generation techniques, such as using conditioning variables, to steer the model's outputs towards the desired attributes or topics.
Human-in-the-Loop: I incorporated human feedback in the training loop, allowing domain experts to review and correct outputs, which helped the model learn from real-world corrections and improve its performance.

5. Ethical and Bias Considerations
Challenge:

Generative models can inadvertently produce biased, offensive, or harmful content, reflecting biases present in the training data. Ensuring ethical use and mitigating biases is crucial.
Solution:

Bias Detection and Mitigation: I implemented techniques to detect and mitigate biases in the training data and model outputs. This included using fairness metrics, adversarial debiasing, and fine-tuning with balanced datasets.
Content Filtering: I deployed content filtering mechanisms to identify and block potentially harmful or inappropriate outputs. This involved using rule-based filters, toxicity classifiers, and manual review processes.
Transparency and Accountability: I maintained transparency by documenting the data sources, training processes, and limitations of the models. Additionally, I established accountability measures by involving diverse teams in the development and review stages.

6. Integration and Deployment
Challenge:

Integrating generative models into existing systems and deploying them in a production environment can be complex, involving considerations around latency, scalability, and maintainability.
Solution:

Microservices Architecture: I used a microservices architecture to modularize the generative AI components, making them easier to integrate, scale, and maintain independently of other system components.
API Development: I developed robust APIs to interact with the generative models, ensuring they could be easily consumed by other services and applications. This included implementing rate limiting, caching, and load balancing to handle varying traffic loads.
Continuous Monitoring: I set up continuous monitoring and logging mechanisms to track the model's performance and detect any issues in real-time. This allowed for quick identification and resolution of problems, ensuring reliable operation in production.
By addressing these challenges with strategic solutions, I have been able to successfully implement and deploy generative AI models that deliver meaningful and impactful results in various applications.

Prompt

How to preprocess large-scale datasets for AI applications?

Response

Preprocessing large-scale datasets for AI applications involves several crucial steps to ensure the data is clean, consistent, and suitable for training models. Here are the steps typically involved in preprocessing large-scale datasets:

1. Data Collection and Ingestion
Goal: Gather data from various sources and ensure it's properly stored for processing.

Data Sources: Collect data from databases, APIs, web scraping, sensors, and other sources.
Storage Solutions: Use distributed storage systems like Hadoop Distributed File System (HDFS), cloud storage (AWS S3, Google Cloud Storage), or data warehouses (BigQuery, Redshift).
2. Data Exploration and Initial Cleaning
Goal: Understand the dataset and perform initial cleaning.

Data Inspection: Use tools like Pandas, Dask, or Apache Spark to load and inspect the data. Check for missing values, outliers, and data types.
Initial Cleaning: Remove duplicates, handle missing values (e.g., impute or drop), and filter out irrelevant data.
3. Data Formatting and Normalization
Goal: Standardize the data to ensure consistency.

Data Types: Convert data to appropriate types (e.g., integers, floats, dates).
Normalization: Scale numerical features to a common range using techniques like Min-Max scaling or Standardization (Z-score normalization).
Categorical Encoding: Convert categorical variables to numerical format using techniques like one-hot encoding, label encoding, or embeddings for high-cardinality features.
4. Handling Missing Data
Goal: Address any missing data points appropriately.

Imputation: Use statistical methods (mean, median, mode) or machine learning models to estimate missing values.
Deletion: If the proportion of missing data is small, rows or columns with missing values can be removed.
5. Data Augmentation and Transformation
Goal: Enhance the dataset with additional features or transformations.

Feature Engineering: Create new features from existing data (e.g., extracting date parts, combining features).
Data Augmentation: For image or text data, use techniques like rotation, flipping, noise addition, or synthetic data generation to increase the diversity of the training set.
6. Data Splitting
Goal: Divide the dataset into training, validation, and test sets.

Random Split: Randomly split the dataset into training, validation, and test sets (e.g., 70-20-10 split).
Stratified Split: Ensure that the splits maintain the distribution of key features (e.g., class labels) using stratified sampling.
7. Distributed Processing
Goal: Efficiently process large-scale data using distributed computing.

Frameworks: Use distributed computing frameworks like Apache Spark, Dask, or Hadoop to process data in parallel.
Chunking: Process the data in chunks or batches to handle memory constraints and improve performance.
8. Data Validation and Quality Checks
Goal: Ensure the processed data meets quality standards.

Validation Rules: Implement rules to check data consistency, integrity, and accuracy (e.g., range checks, format checks).
Automated Testing: Use automated scripts to validate the processed data and detect anomalies.
9. Data Storage and Versioning
Goal: Store the processed data securely and maintain versions.

Storage: Store the processed data in databases, data lakes, or file systems for easy access.
Versioning: Use tools like DVC (Data Version Control) or Delta Lake to manage data versions and track changes.
Example: Preprocessing a Large-Scale Text Dataset
Data Collection:

python
Copy code
import pandas as pd
import dask.dataframe as dd

# Load data from CSV using Dask for large-scale processing
df = dd.read_csv('large_text_dataset.csv')
Data Inspection and Cleaning:

python
Copy code
# Inspect the data
print(df.head())

# Remove duplicates
df = df.drop_duplicates()

# Handle missing values
df = df.dropna(subset=['text'])
Normalization and Encoding:

python
Copy code
# Tokenization and encoding of text data
from sklearn.preprocessing import LabelEncoder
import nltk
nltk.download('punkt')

# Tokenize text
df['tokenized_text'] = df['text'].apply(nltk.word_tokenize)

# Encode labels
le = LabelEncoder()
df['label'] = le.fit_transform(df['category'])
Data Splitting:

python
Copy code
from sklearn.model_selection import train_test_split

train_df, test_df = train_test_split(df, test_size=0.2, stratify=df['label'])
train_df, val_df = train_test_split(train_df, test_size=0.25, stratify=train_df['label'])  # 0.25 x 0.8 = 0.2
Distributed Processing:

python
Copy code
# Process data in chunks using Dask
def preprocess_chunk(chunk):
    # Example preprocessing function
    chunk['clean_text'] = chunk['text'].str.lower()
    return chunk

processed_df = df.map_partitions(preprocess_chunk)
Data Storage and Versioning:

python
Copy code
# Save the processed data
processed_df.to_csv('processed_text_dataset.csv', index=False)

# Use DVC for version control
# dvc add processed_text_dataset.csv
By following these steps, you can preprocess large-scale datasets efficiently, ensuring they are ready for AI model training and deployment.

Prompt

9: How to fine-tune the Azure OpenAI's GPT-4 model?

Response

Fine-tuning Azure OpenAI's GPT-4 model involves adapting the pre-trained model to perform specific tasks by training it further on your own dataset. Here’s a step-by-step guide on how to fine-tune the GPT-4 model on Azure:

Step 1: Set Up Your Azure Environment
Create an Azure Account:

Sign up for an Azure account if you don’t already have one.
Set Up Azure OpenAI Service:

Navigate to the Azure portal.
Create a new resource and search for "Azure OpenAI."
Create an instance of the Azure OpenAI service.
Step 2: Prepare Your Data
Format Your Dataset:

Ensure your dataset is in a suitable format (e.g., JSON, CSV) with input-output pairs that the model can learn from.
Example format for text generation:
json
Copy code
[
    {"prompt": "Translate the following English text to French: 'Hello, how are you?'", "completion": "Bonjour, comment ça va?"},
    {"prompt": "Translate the following English text to French: 'Good morning'", "completion": "Bon matin"}
]
Clean and Preprocess Data:

Remove any irrelevant information, handle missing values, and ensure data consistency.
Tokenize text if necessary, but generally, Azure OpenAI handles tokenization internally.
Step 3: Upload Your Data to Azure Storage
Create a Storage Account:

In the Azure portal, create a new storage account if you don’t have one.
Upload Your Dataset:

Use Azure Blob Storage to upload your dataset.
Navigate to your storage account -> Blob service -> Containers -> Create a new container -> Upload your data files.
Step 4: Configure and Run Fine-Tuning
Install Azure CLI:

Install the Azure Command-Line Interface (CLI) if you haven’t already.
Configure the CLI with your Azure account: az login
Prepare Fine-Tuning Script:

Create a script or use Azure’s interface to start the fine-tuning process.
Example Python script using Azure SDK:
python
Copy code
from azure.identity import DefaultAzureCredential
from azure.ai.openai import OpenAIClient, FineTuneJobParameters

# Authenticate
credential = DefaultAzureCredential()
client = OpenAIClient(endpoint="https://<your-openai-endpoint>.cognitiveservices.azure.com/", credential=credential)

# Define fine-tune job parameters
fine_tune_job = FineTuneJobParameters(
    training_file_url="<blob-storage-url-to-your-training-dataset>",
    model="gpt-4",
    compute_target="Standard_DS3_v2",
    n_epochs=4,
    batch_size=8
)

# Start fine-tune job
fine_tune_job = client.begin_create_fine_tune_job(fine_tune_job)
fine_tune_job.wait()
print(f"Fine-tune job completed: {fine_tune_job.result()}")
Run Fine-Tuning Job:

Execute the script or use the Azure portal to submit the fine-tuning job.
Monitor the progress through the Azure portal or by checking job status programmatically.
Step 5: Deploy the Fine-Tuned Model
Deploy the Model:

Once fine-tuning is complete, deploy the fine-tuned model to a web service endpoint.
Example using Azure CLI:
sh
Copy code
az openai model deploy --model-name <fine-tuned-model-name> --resource-group <your-resource-group> --service-name <your-openai-service-name>
Test the Deployment:

Use the endpoint to test the model’s performance with new prompts and verify that it behaves as expected.
Example using Python:
python
Copy code
from azure.identity import DefaultAzureCredential
from azure.ai.openai import OpenAIClient

# Authenticate
credential = DefaultAzureCredential()
client = OpenAIClient(endpoint="https://<your-openai-endpoint>.cognitiveservices.azure.com/", credential=credential)

# Test the deployed model
response = client.completion(
    prompt="Translate the following English text to French: 'I love programming.'",
    model="<fine-tuned-model-name>"
)
print(response.choices[0].text)
Step 6: Monitor and Maintain the Model
Monitor Performance:

Continuously monitor the model’s performance and usage through Azure Metrics and Logs.
Adjust the model or retrain as necessary based on performance feedback.
Update and Improve:

Regularly update the dataset with new data and re-run fine-tuning to keep the model current.
Implement feedback loops to capture user input and improve model accuracy and relevance.
By following these steps, you can successfully fine-tune and deploy the Azure OpenAI GPT-4 model to suit your specific application needs.

Prompt

14. What evaluation metrics to use to assess the quality and relevance of generative AI outputs?

Response

Evaluating the quality and relevance of generative AI outputs is crucial for ensuring that the models meet the desired performance and application requirements. Different tasks and domains may require specific metrics, but some commonly used evaluation metrics include:

1. Perplexity
Purpose:

Measures how well a probabilistic model predicts a sample.
Application:

Used in language modeling to assess the model's ability to predict text sequences.
Interpretation:

Lower perplexity indicates better performance.
2. BLEU (Bilingual Evaluation Understudy)
Purpose:

Measures the similarity between generated text and reference text.
Application:

Commonly used in machine translation, summarization, and text generation.
Interpretation:

Higher BLEU scores indicate better performance, but it's sensitive to exact word matches.
3. ROUGE (Recall-Oriented Understudy for Gisting Evaluation)
Purpose:

Measures the overlap of n-grams between generated and reference text.
Application:

Frequently used in summarization and text generation.
Interpretation:

Higher ROUGE scores indicate better performance, focusing on recall and precision.
4. METEOR (Metric for Evaluation of Translation with Explicit ORdering)
Purpose:

Considers synonymy, stemming, and word order.
Application:

Used in machine translation and text generation.
Interpretation:

Higher METEOR scores indicate better performance and flexibility in language use.
5. CIDEr (Consensus-based Image Description Evaluation)
Purpose:

Measures the consensus between generated captions and multiple reference captions.
Application:

Primarily used in image captioning.
Interpretation:

Higher CIDEr scores indicate better performance.
6. SPICE (Semantic Propositional Image Caption Evaluation)
Purpose:

Measures how well image captions align with scene graphs (semantic content).
Application:

Used in image captioning.
Interpretation:

Higher SPICE scores indicate better alignment with semantic content.
7. F1 Score
Purpose:

Combines precision and recall into a single metric.
Application:

Used in classification tasks, including named entity recognition and sentiment analysis.
Interpretation:

Higher F1 scores indicate better balance between precision and recall.
8. Human Evaluation
Purpose:

Measures subjective aspects like coherence, relevance, and fluency.
Application:

Used across various text generation tasks.
Interpretation:

Human evaluators rate the quality of outputs, providing insights into aspects that automated metrics may miss.
9. Embedding-based Metrics (e.g., BERTScore)
Purpose:

Measures semantic similarity using pre-trained language model embeddings.
Application:

Used in text generation and translation.
Interpretation:

Higher scores indicate better semantic similarity between generated and reference texts.
10. Diversity Metrics
Purpose:

Measures the variety in generated outputs.
Application:

Important in creative tasks like story generation and dialogue systems.
Interpretation:

Metrics like distinct-n (number of distinct n-grams) indicate higher diversity.
11. Coverage
Purpose:

Measures how much of the reference content is covered by the generated output.
Application:

Used in summarization and text generation.
Interpretation:

Higher coverage indicates that the generated output includes more of the important content from the reference.
12. Cosine Similarity
Purpose:

Measures the cosine of the angle between two vectors (e.g., document embeddings).
Application:

Used in various text similarity tasks.
Interpretation:

Higher cosine similarity indicates higher similarity between the texts.
Choosing the Right Metrics
Task-Specific Requirements: Different tasks (e.g., translation, summarization, image captioning) require different metrics.
Balanced Evaluation: Use a combination of automated metrics and human evaluation to capture different aspects of quality.
Consistency: Ensure that the chosen metrics align with the goals of the application and provide meaningful insights into model performance.
Example: Evaluating a Text Generation Model
Suppose you have developed a text generation model and want to evaluate its performance:

Automated Metrics:

BLEU: Evaluate the n-gram overlap with reference texts.
ROUGE: Measure the recall and precision of n-grams.
BERTScore: Assess semantic similarity using pre-trained BERT embeddings.
Human Evaluation:

Fluency: Rate the grammatical correctness and naturalness of the text.
Relevance: Rate how well the generated text adheres to the input prompt or context.
Coherence: Rate the logical flow and consistency of the text.
Diversity:

Distinct-n: Calculate the proportion of unique n-grams to assess diversity in the generated outputs.
By combining these metrics, you can comprehensively evaluate the quality and relevance of generative AI outputs and ensure that your models perform well across different dimensions.

Prompt

Can you write some code to demonstrate Perplexity for generation via Gemini Flash model?

Response

To demonstrate how to calculate perplexity for text generation using a language model like Gemini Flash, you would typically need to use a pre-trained model and library that supports this functionality. Since Gemini Flash might not have direct support in libraries like Hugging Face Transformers or PyTorch, I'll provide an example using a generic transformer model from Hugging Face's transformers library. You can adapt this example to work with Gemini Flash or any other specific model if necessary.

Here's how you can compute perplexity using the Hugging Face transformers library and the PyTorch framework:

Code Example to Compute Perplexity
Install Required Libraries:

sh
Copy code
pip install transformers torch
Code to Compute Perplexity:

python
Copy code
import torch
from transformers import GPT2Tokenizer, GPT2LMHeadModel
import math

# Load pre-trained model and tokenizer
model_name = 'gpt2'  # Replace with 'gemini-flash' or other model if available
tokenizer = GPT2Tokenizer.from_pretrained(model_name)
model = GPT2LMHeadModel.from_pretrained(model_name)

# Ensure the model is in evaluation mode
model.eval()

def compute_perplexity(text):
    # Tokenize input text
    inputs = tokenizer(text, return_tensors='pt')
    input_ids = inputs['input_ids']
    
    # Calculate log-likelihood of the input text
    with torch.no_grad():
        outputs = model(input_ids, labels=input_ids)
        log_likelihood = outputs.loss.item() * input_ids.size(1)
    
    # Compute perplexity
    perplexity = math.exp(log_likelihood / input_ids.size(1))
    return perplexity

# Example text
text = "Hello, how are you today?"

# Compute perplexity for the example text
perplexity = compute_perplexity(text)
print(f"Perplexity: {perplexity}")
Explanation:
Load Model and Tokenizer:

We load a pre-trained GPT-2 model and tokenizer from Hugging Face's model hub. Replace 'gpt2' with the appropriate model name if using Gemini Flash or any other model.
Tokenize Text:

The input text is tokenized into input IDs using the tokenizer.
Compute Log-Likelihood:

The model computes the loss, which is the negative log-likelihood. We multiply the loss by the sequence length to get the total log-likelihood for the entire input text.
Compute Perplexity:

Perplexity is calculated as the exponentiation of the average log-likelihood per token. This gives a measure of how well the model predicts the text, with lower perplexity indicating better performance.
Adapting to Gemini Flash:
If you're specifically using Gemini Flash or another model not directly supported by transformers, you would need to adapt the code to use the appropriate library or API provided by the model's framework. The core idea remains the same: tokenize the input, compute the log-likelihood, and derive the perplexity.

Feel free to adapt the code to suit the specific requirements and libraries associated with the Gemini Flash model or any other generative model you are working with.

Tags: Interview Preparation,Generative AI,Technology,Large Language Models,

Round (2) - 18 Interview Questions on Cypher Queries and Knowledge Graph Using Neo4j (For Data Scientist Role) - Jul 2024

To See All Interview Preparation Articles: Index For Interviews Preparation

THESE QUERIES HAVE BEEN TRIED ON:
Server version:	Neo4j/5.20-aura
Server address:	fd39c7f2.databases.neo4j.io:7687

QUESTION 1:

HOW DO YOU CREATE AN INDEX ON STRING TYPE DATA IN NEO4J?

ANSWER:

(1)

Ref

Question from site:

I have a NextJS app using a Neo4j DB, in which I want to create a index like this:

CALL db.index.fulltext.createNodeIndex(
  "FTS_Person_Company_fullName",
  ["Person", "Company"],
  ["fullName"],
  {analyzer: "standard-folding", eventually_consistent: "true"})

Answer from site:

In neo4j 5.x, the Cypher language directly supports creating fulltext indexes.

For example, in your case:

CREATE FULLTEXT INDEX FTS_Person_Company_fullName 
FOR (n:Person|Company)
ON EACH [n.fullName]
OPTIONS {
  indexConfig: {
    `fulltext.analyzer`: "standard-folding",
    `fulltext.eventually_consistent`: true
  }
}

Ref (2)

CALL db.index.fulltext.createNodeIndex('documents', ['Document'], ['title','text'])

~~~

QUESTION 2:

IF I WANT TO CREATE TO CREATE AN INDEX LIKE THIS:
CALL db.index.fulltext.createNodeIndex('MY_FTS_EPICS', ['EPIC'], ['SUMMARY','DESCRIPTION', 'ACCEPTANCE_CRITERIA'])

WHAT WOULD BE EQUIVALENT 'CREATE FULLTEXT INDEX' QUERY:


ANSWER 2:

Ref

CREATE FULLTEXT INDEX MY_FTS_EPICS 
FOR (n:EPIC)
ON EACH [n.SUMMARY, n.DESCRIPTION, n.ACCEPTANCE_CRITERIA]

~~~

QUESTION 3:

WHY ARE WE DOING IT WITH 'CREATE FULLTEXT INDEX' QUERY:::

INSTEAD OF SOME THIS LIKE THIS:
CALL db.index.fulltext.createNodeIndex('MY_FTS_EPICS', ['EPIC'], ['SUMMARY','DESCRIPTION', 'ACCEPTANCE_CRITERIA'])

ANSWER:

Because createNodeIndex() is not supported in Neo4j AuraDB.

Neo.ClientError.Procedure.ProcedureNotFound
There is no procedure with the name `db.index.fulltext.createNodeIndex` registered for this database instance. Please ensure you've spelled the procedure name correctly and that the procedure is properly deployed.

~~~

QUESTION 4:

WRITE A QUERY TO FETCH RESULTS FOR INPUT STRING 'SGQ' FOR THE INDEX (MY_FTS_EPICS) THAT YOU HAVE CREATED ABOVE.

ANSWER:

CALL db.index.fulltext.queryNodes('MY_FTS_EPICS', 'SGQ') 
YIELD node, score 
RETURN node, score

Ref 1
Ref 2
Ref 3

~~~

QUESTION 5:

What would you get if you pass NODE LABEL in the above query in place of INDEX name?

ANSWER:

We get an error:


CALL db.index.fulltext.queryNodes('EPIC', 'My SGQ and yours') YIELD node, score
RETURN node, score


Neo.ClientError.Procedure.ProcedureCallFailed
Failed to invoke procedure `db.index.fulltext.queryNodes`: Caused by: java.lang.IllegalArgumentException: There is no such fulltext schema index: EPIC

~~~

QUESTION 6:

Can you combine calls to queryNodes() with a WHERE clause as in a relational query?

ANSWER:

Short answer: No.

Ref

"I understand now that since the full-text-search is based on the Lucene index it can't be made on a subset of nodes."

~~~

QUESTION 7:

What all string comparison operators do you know?

Answer:

Ref

STRING-specific comparison operators

	STARTS WITH, 
	ENDS WITH, 
	CONTAINS, 
	=~ (regex matching)

~~~

QUESTION 8:

Are there more operators for or related to strings?

Answer:

Ref

Yes. 

The string operators comprise:

	concatenating STRING values: + and ||

	checking if a STRING is normalized: IS NORMALIZED

~~~

QUESTION 9:

What does 'IS NORMALISED' do?

Answer:

Checking if a STRING IS NORMALIZED
Introduced in 5.17
The IS NORMALIZED operator is used to check whether the given STRING is in the NFC Unicode normalization form:

Unicode normalization is a process that transforms different representations of the same string into a standardized form. For more information, see the documentation for Unicode normalization forms.

Query
RETURN "the \u212B char" IS NORMALIZED AS normalized

Table 14. Result

normalized
false

Because the given STRING contains a non-normalized Unicode character (\u212B), false is returned.

~~~

QUESTION 10:

What all string related functions are available in Neo4j?
Which all have you used?

Answer:

Ref

btrim()
btrim() returns the original STRING with leading and trailing trimCharacterString characters removed. If trimCharacterString is not specified then all leading and trailing whitespace will be removed.

left()
left() returns a STRING containing the specified number of leftmost characters of the given STRING.

ltrim()
ltrim() returns the original STRING with leading trimCharacterString characters removed. As of Neo4j 5.20, a trimCharacterString can be specified. If this is not specified all leading whitespace will be removed.

normalize()
normalize() returns the given STRING normalized using the NFC Unicode normalization form.

Unicode normalization is a process that transforms different representations of the same string into a standardized form. For more information, see the documentation for Unicode normalization forms.

The normalize() function is useful for converting STRING values into comparable forms. When comparing two STRING values, it is their Unicode codepoints that are compared. In Unicode, a codepoint for a character that looks the same may be represented by two, or more, different codepoints. For example, the character < can be represented as \uFE64 (﹤) or \u003C (<). To the human eye, the characters may appear identical. However, if compared, Cypher® will return false as \uFE64 does not equal \u003C. Using the normalize() function, it is possible to normalize the codepoint \uFE64 to \u003C, creating a single codepoint representation, allowing them to be successfully compared.


normalize(), with specified normal form
normalize() returns the given STRING normalized using the specified normalization form. The normalization form can be of type NFC, NFD, NFKC or NFKD.

replace()
replace() returns a STRING in which all occurrences of a specified STRING in the given STRING have been replaced by another (specified) replacement STRING.

reverse()
reverse() returns a STRING in which the order of all characters in the given STRING have been reversed.

right()
right() returns a STRING containing the specified number of rightmost characters in the given STRING.

rtrim()
rtrim() returns the original STRING with trailing trimCharacterString characters removed. As of Neo4j 5.20, a trimCharacterString can be specified. If this is not specified all trailing whitespace will be removed.

split()
split() returns a LIST resulting from the splitting of the given STRING around matches of the given delimiter.

substring()
substring() returns a substring of the given STRING, beginning with a zero-based index start and length.

toLower()
toLower() returns the given STRING in lowercase.

toString()
toString() converts an INTEGER, FLOAT, BOOLEAN, STRING, POINT, DURATION, DATE, ZONED TIME, LOCAL TIME, LOCAL DATETIME or ZONED DATETIME value to a STRING.
This function will return an error if provided with an expression that is not an INTEGER, FLOAT, BOOLEAN, STRING, POINT, DURATION, DATE, ZONED TIME, LOCAL TIME, LOCAL DATETIME or ZONED DATETIME value.

toStringOrNull()
The function toStringOrNull() converts an INTEGER, FLOAT, BOOLEAN, STRING, POINT, DURATION, DATE, ZONED TIME, LOCAL TIME, LOCAL DATETIME or ZONED DATETIME value to a STRING.
If the expression is not an INTEGER, FLOAT, BOOLEAN, STRING, POINT, DURATION, DATE, ZONED TIME, LOCAL TIME, LOCAL DATETIME or ZONED DATETIME value, null will be returned.

toUpper()
toUpper() returns the given STRING in uppercase.

trim()
trim() returns the given STRING with the leading and/or trailing trimCharacterString character removed. As of Neo4j 5.20, a trimCharacterString can be specified. If this is not specified all leading and/or trailing whitespace will be removed.

~~~

QUESTION 11:

STRINT matching in Neo4j using STRING COMPARISON operators is case-sensitive or case-insensitive?

ANSWER:

Short answer: case-sensitive

Ref

STRING matching
The prefix and suffix of a STRING can be matched using STARTS WITH and ENDS WITH. To undertake a substring search (that is, match regardless of the location within a STRING), use CONTAINS.

The matching is case-sensitive. Attempting to use these operators on values which are not STRING values will return null.

~~~

QUESTION 12:

Write the query to find out ENTITY(s) where SGQ appears in it's NAME property.

ANSWER:

MATCH (n:ENTITY) 
WHERE n.NAME CONTAINS 'SGQ'
RETURN n.NAME;

~~~

QUESTION 13:
WRITE THE ABOVE QUERY USING REGEX:

Answer:
MATCH (n:ENTITY) 
WHERE n.NAME =~ '.*SGQ.*'
RETURN n.NAME;

~~~

QUESTION 14:

WRITE THE ABOVE QUERY USING FULLTEXT INDEX:

Answer:

CREATE FULLTEXT INDEX MY_FTS_ON_ENTITY 
FOR (n:ENTITY)
ON EACH [n.NAME]

CALL db.index.fulltext.queryNodes('MY_FTS_ON_ENTITY', 'SGQ') 
YIELD node, score 
RETURN node.NAME, score



1
"SGQ"
2.5313024520874023

2
"ABANDON SGQ"
2.0718064308166504

3
"SGQ output file"
1.7535011768341064

4
"SGQ quoting process"
1.7535011768341064

5
"SGQ Quote PDF"
1.7535011768341064

6
"BPR-SGQ-ISG Refresh"
1.5199768543243408

~~~

QUESTION 15:
Which query runs faster one that uses 'CONTAINS', one that uses "REGEX" or one that uses db.index.fulltext.queryNodes()?
An example of query: 
MATCH (n:ENTITY) 
WHERE n.NAME CONTAINS 'SGQ'
RETURN n.NAME;

And what one is best to use?

Answer:

All the three queries run in a comparable times.

Time for CONTAINS:
# 71.3 ms ± 2.52 ms per loop
# 70.6 ms ± 635 µs per loop

Time for REGEX:
# 72 ms ± 2.32 ms per loop
# 70.6 ms ± 643 µs per loop

Time for fulltext.QUERYNODES():
# 70.8 ms ± 601 µs per loop
# 70.1 ms ± 393 µs per loop


The advantage is in using the fulltext.queryNodes() because it also returns a SCORE to figure out which is the best match, or which is the worst match.

'SGQ'	2.5313024520874023
'ABANDON SGQ'	2.0718064308166504
'SGQ output file'	1.7535011768341064
'SGQ quoting process'	1.7535011768341064
'SGQ Quote PDF'	1.7535011768341064
'BPR-SGQ-ISG Refresh'	1.5199768543243408

~~~

QUESTION 16:

Match a node in Neo4j from a couple options for it's label. 

Ref

I want to match a graph where a node can be typeX or typeY... 

###
One way is

MATCH (n) WHERE labels(n) IN ['typeX','typeY'] 
WITH n
MATCH (n)-[]-(z)
RETURN z
However, if "either typeX or typeY" are queried frequently and share some common purpose in your domain, you could add another common label to them like "commonXY" and query using that label instead.

--------------------------------------------------------------------------

MATCH (x:ENTITY) WHERE x.NAME IN ['SGQ','ABANDON SGQ'] 
WITH x
MATCH (x)-[:APPEARS_IN_EPIC]->(z:EPIC)<-[:APPEARS_IN_EPIC]-(y:ENTITY)
WHERE y.NAME in ['SGQ quoting process']
RETURN x, y, z

OUTPUT:

(:ENTITY {NAME: SGQ})	
(:ENTITY {NAME: SGQ quoting process})	
(:EPIC {SUMMARY: BPR-QTE-Dental Rate Bank updated to support DHMO on SGQ output file, DESCRIPTION: As an agent quoting DHMO Dental Plans in SGQ, _x000D_@@NL@@I want to be able to update the rates for these plans, _x000D_@@NL@@So that the price for these dental plans will be more compelling to the small group employer. , ACCEPTANCE_CRITERIA: Problem Statement _x000D_Current dental rate bank in SGQ does not support DHMO planss. _x000D__x000D_Benefit Hypothesis _x000D_By allowing DHMO plan rates to be updated through Rate Bank in SGQ, ElevanceHealth can present more competitive prices for the selected dental rate plans. _x000D__x000D_Process Flow (Scenario - new sales) _x000D_Given an agent has successfully accessed SGQ, _x000D_And the agent is quoting DHMO dental plans for a small group employer. _x000D_After proceeding through the SGQ quoting process as designed, _x000D_And the agent is on the quotes tab to see the plans selected to be quoted.  _x000D_When reviewing the DHMO plans there will be an 'override' link where the agent will be able to update the rates for the selected DHMO plans., KY: BPP-149271, CUSTOM_ID: 2})

~~~

QUESTION 17:
How do set a property on all nodes with label EPIC?

Ref

MATCH (n:EPIC)
SET n.GeneratedFrom = 'JIRA', n.CreatedOn = date("2024-07-05")
RETURN n

# ALSO CHECK:
# Set multiple properties using one SET clause
MATCH (n {name: 'Andy'})
SET n.position = 'Developer', n.surname = 'Taylor'

# ALSO CHECK:
https://neo4j.com/docs/getting-started/cypher-intro/dates-datetimes-durations/
# Creating and updating Date and Datetime type values

MATCH (article:Article {title: "Dates, Datetimes, and Durations in Neo4j"})
SET article.datePublished = date("2019-09-30")

# HOW DO YOU SET A PROPERTY ON ALL NODES WITH LABEL EPIC OR STORY?

MATCH (n) 
WHERE n:EPIC OR n:STORY
SET n.GeneratedFrom = 'JIRA', n.CreatedOn = date("2024-07-05")
RETURN n

# ALSO CHECK
Update a property:
SET can be used to update a property on a node or relationship. This query forces a change of type in the age property:

Query
MATCH (n {name: 'Andy'})
SET n.age = toString(n.age)
RETURN n.name, n.age

MATCH (n:EPIC)
WHERE n.GeneratedFrom = 'Bot'
SET n.CreatedOn = date(n.CreatedOn)
RETURN n;

~~~

QUESTION 18:
RETURN ALL NODES WITH LABEL 'EPIC' WITH PROPERTY GeneratedFrom = 'Bot'

ANSWER:

MATCH (n:EPIC)
WHERE n.GeneratedFrom = 'Bot'
RETURN n;

...

# ALSO CHECK THE ASSOCIATED DELETE QUERY

# Delete a node with all its relationships
MATCH (n:EPIC)
WHERE n.GeneratedFrom = 'Bot'
DETACH DELETE n;

If you would use just 'DELETE n' instead of 'DETACH DELETE n', following error would come:
Neo.ClientError.Schema.ConstraintValidationFailed
Cannot delete node<555>, because it still has relationships. To delete this node, you must first delete its relationships.

Tags: Database,Technology